A featured contribution from Leadership Perspectives: a curated forum reserved for leaders nominated by our subscribers and vetted by the Construction Tech Review Advisory Board.
Fullbeauty Brands
Thinking Out-Of-The-Box To Launch Business Beyond Borders
Bill Bass
At FULLBEAUTY Brands, we test extensively to distinguish technologies that matter from ones that don’t. Because new website features tend to be used by the best customers, nearly all features will show a higher conversion rate or revenue per visit due to selection bias. The best way to truly test a new technology is an A/B test: show half a random sample the new technology, hide it from the other half, then compare the performance of the two groups. We wanted to share the results of two recent technology tests. One validated a vendor’s claims; the other uncovered a big waste of money
People have a range of body shapes that can resemble an apple, pear, hourglass, or inverted triangle, and matching body shapes with apparel sizes and fits that can vary widely across brands is a hard solve for ecommerce.
The best way to truly test a new technology is an a/b test: show half a random sample the new technology, hide it from the other half, then compare the performance of the two groups
We started with an A/B test and found no difference in the dollars generated during a visit. Obviously not a good initial result for the vendor. But in addition to the initial visit, we wanted to know the downstream impact on metrics such as return rates and lifetime value of the customer. We conducted a matched cohort analysis to account for selection bias and tracked the performance of TrueFit users over the next 100 days.
The result, despite a slight increase in return rates from the TrueFit cohort -- which we theorize results from higher expectations for improved fit -- was a 17 percent increase in dollars per customer. TrueFit failed the A/B test and actually increased return rates, but the overall downstream impact of the technology was highly positive. We don’t run across many technologies that generate a 17 percent improvement in business performance. This was a pleasant surprise.
Another technology we tested was retargeting site visitors with display ads on other websites. This is a popular way to spend digital advertising dollars because the return on investment appears to be extremely high. We wanted to isolate the incremental impact of the advertising to see if we were wasting ad dollars on customers who were already going to return and complete their purchase. The basic rule is to avoid handing out pizza coupons to customers already in the line at the pizza shop.
Our A/B test showed the retargeting ads had literally zero impact on customer behavior. The revenue per visitor from both samples was identical. There is an old adage “half of advertising doesn’t work, we just don’t know which half”. Testing discovered retargeting ads fall into the half that doesn’t work.
