My name is Bryan Mahoney, and I'm the Co-founder and Chief Technology Officer here at arfa.
On Developing Mature Product Metrics
So as a company matures, as my eCommerce stack grows, as the amount of product that we've shipped a customer, we have more surface area. We remain really focused on those key metrics. Like they still exist. I think what's most interesting is that we now have something that we can benchmark against.
We understand that we have a track record of a year or two or three of data. We understand how AOV has evolved, how we've been able to introduce certain changes into the platform and how that's affected things like average basket size. But now, we get access to even more sophisticated metrics; How long does it take a customer to come back and order again? How likely is a customer to tell a friend about the brand? So, you know, now I'm talking about referrals. How likely are they to leave a
review? How positive is that review going to be? What sort of product can I build that will influence any of those outcomes?
Typically the way that I measure the effectiveness of any changes that we're making to product at a certain amount of scale is with A/B testing. And A/B testing done right requires a reasonably significant investment for any team to make and you want to make sure that if you're investing that amount of time and money and effort in A/B testing, that you've got really solid metrics that you can rely on to give you signal as to which direction you ultimately want to go in.
On Where Teams Often Fall Short
What's been so fascinating about the eCommerce landscape over the last, say five to seven years, has been an increasing focus on data. And we can be paralyzed by data, we're collecting more data than we ever have before. We can fall in love with the idea of testing our way into everything and the only way that we move forward is with an A/B test, and I think you need to realize the stage at which your company is at. I would always advocate for younger brands to take more chances, so you don't need to test your way into everything. You need to establish those solid benchmarks.
If I'm in the marketing department and I am defining revenue in a certain way, but I'm in the finance department and I'm defining revenue in a completely different way, and we're both very focused on data, then at a certain point we're gonna feel some pain - there's going be a disconnect there. Some common pitfalls are being very focused on data but not speaking the same language. So how do we think about ways that we can bring everyone together so that we're ultimately looking at the same thing and we're focusing our efforts in the right place? If only there was a platform... [Yaguara]
