This is a guest blog post by Optimizely, a Demandware premier LINK partner

Almost a century ago, the pioneering merchant John Wanamaker complained that, “half of the money I spend on advertising is wasted. The trouble is, I don’t know which half.”

Perhaps if Wanamaker were able to test promotions ahead of time, he would have had a better handle on what worked and what didn’t. Thankfully, today, robust testing and analytics tools remove a lot of the guesswork, and help retailers accurately predict what will drive revenue and repeat business.

Many retailers have seen a disproportionate increase in costs in competing for online sales, and as a result have been forced to seek efficiencies and optimize their sites to raise their chances of driving profit and revenue. In an earlier post, Demandware advocated a shift away from conversion rate as the focal performance metric of an ecommerce site, pointing out that, “conversion rate has (at best) lost relevance, and at worst is misleading and harmful to your reporting.”

A story from Brooks Running highlights this shift. In addition to impacting conversion rate and revenue, the team at Brooks wanted to see how they could impact return behavior, which has an impact on margin and the customer experience. After mining returns data for opportunities, they ran A/B tests to show different customer service messaging when a customer indicates high potential for return such as adding two shoes within half a size. Their testing efforts successfully resulted in an 80% reduction in order return rate for certain order types. (Optimizely will host a breakout session with Brooks on this topic, “Optimizing beyond conversion rate,” on April 6th at XChange)

Trunk Club, a subscription-based clothing online service, offers another example. The company was challenged to scale their white glove personalized service in a cost-effective way, and leveraged their cross-functional optimization team to tackle the issue. By running experiments to separate browsing shopper behavior from shoppers displaying strong purchase intent, Trunk Club improved stylist efficiency by 90%.

How? Trunk Club used Optimizely testing to validate its theory that letting shoppers “heart” an outfit or garment would increase engagement and increase stylist efficiency.

Engaging team members across sales, product, merchandising, creative and design was a critical part of their success in improving operational productivity and profitability of the service. “Whenever we’re curious, confused or in doubt, we run a test and let the data speak for itself,” says Mike Wolf, Trunk Club’s Product Design Lead.

In theory, retailers could test every granular component of their Website and marketing efforts – color and font size, layout of a product grid – and in fact the Demandware Retail Practice team are experts in identifying seemingly minute changes that can have a big impact.

Changing shopper preferences and behaviors are forcing retailers to adapt at a dizzying pace. For pure-play ecommerce site Revolve, every engagement presents an opportunity to lose that customer, because “she’s likely to move on to the next app or site vying for her attention,” if she’s not having a seamless experience, says Grace Hong, Revolve’s Vice President of Product and Design. Optimization has been critical to Revolve’s ability to deliver the best experience to the customer at every touchpoint whether desktop, mobile, and email.

“It’s absolutely necessary to show her what she wants as soon as she comes to us, even if she doesn’t yet know what that item is.” One example: Continuous testing of their “Hot List” page and desktop navigation has been key to converting browsers to buyers, increasing engagement on product detail pages by 20%.

In addition to engaging their shopper on the desktop, Revolve has found tremendous value from growing their email subscriber list to 1.5 million subscribers. The team A/B tested different email sign up experiences to increase sign-ups by 2.25%, a win for customer lifetime value and loyalty in a tried-and-true channel.

With half of its site traffic coming on mobile devices, Revolve launched native apps for iPhone, iPad and Android. The app is a touchpoint that provides the shopper with an even more personalized, focused experience. By testing different mobile web splash page designs, they increased direct app downloads from the mobile site by a whopping 350%.

A robust optimization strategy can drive impactful business value in both top line growth and profitability, but requires a commitment to change. Cross-functional team ownership and integration across business processes is critical to successful execution. To learn more about the impact that optimization can have on your business, visit Optimizely in the Partner Showcase at XChange.