Everything you need to know about A/B Testing
Boost conversions & revenue through price decreases
How Sheets & Giggle's basic, collection-wide price test unlocked an opportunity to boost conversion and revenue.
Want to learn more about this test? We collaborated with NYU Stern and McGill University to create an academic case study (including a teaching note w/ datasets). Read the case study here.
Sheets & Giggles sells naturally cooling sheets made from eucalyptus fibers that are significantly more eco-friendly to produce than cotton, using 95% less water and 30% less energy in the manufacturing process! The company’s brand is light and humorous (bet you could've guessed it from their name), and they’ve won in their market by providing an exceptional quality product at a very competitive price through their Shopify store.
Experimentation is core to the brand, so when CEO Colin McIntosh heard about Intelligems, he was immediately interested! On testing and experimentation he says:
“Experimentation is super crucial to us and now we do price testing constantly across our products. And the best part about price testing is a lot of times you launch a new product, itʼs a little bit finger to the wind. Youʼre doing competitive analysis, youʼre looking at your competitors, and then youʼre seeing what you can charge. Experimentation will be more important as weʼre coming out with a bunch of new product lines this year.”
The first several years of Sheets & Giggles were all about finding product market fit, perfecting the product, optimizing the website, and finding a customer base and the channels to reach them. With some new funding in hand, the team wanted to focus on nailing the company’s “economic engine” - and a big question was “are we priced correctly?”
With that broad focus, the first tests were broad in scope as well - looking at several price points (both higher and lower) across several products. The first several tests were focused on a wide swath of products - all of the core sheets and sheet sets - and included both higher and lower prices in order to get a full picture of customers’ price elasticity.
For the first test, the team opted to test prices across the sheet collection (including their core sheets & sheet sets). They decided to test three groups: the control group, a group ~15% higher, and a group ~15% lower with traffic split evenly between each group in real time. The percent changes were not exact, as we made sure to round prices (e.g., $193.37 became $195) to maintain consistency with the current pricing.
As customers navigated the site, they received a native experience - and prices were consistent across product types. For example, if a customer saw a higher price for fitted sheets, they were also shown the higher price for the bundle of fitted sheets + flat sheets.
After three weeks, results were in... and interestingly, both test groups (the higher and lower prices) were possible winners versus the control! The higher price group had no statistically lower conversion rate, despite the 15% price increase. Meanwhile, the 15% lower price group actually generated nearly a 40% lift in conversion rate, yielding a 15% increase in revenue.
Price testing doesn't have to be one-and-done. In fact, we find that stores that approach testing with a roadmap and plan to get the most out of price testing are the stores that unlock the most value. Following this initial test, the Sheets & Giggles team continued to test to better understand and refine their pricing:
- They ran subsequent tests with "even lower" and "even higher" price points to further flesh out the elasticity curve.
- Next, they began to test individual products (e.g., only duvet covers; only fitted sheets) to get a fine-grained elasticity for each product type.
- The team then went back to test their free shipping threshold and rates to re-adjust for the new prices.