A survey that changed a product overnight & other tales of data-marketing successes
Data that can be used for marketing purposes can come from all kinds of different sources. We can look at data regarding email (opens, clicks, revenue etc), site browsing behavior, buying frequency, buying behavior (only with coupons, only with free shipping, only full price etc), general analytics, and more.
To get a better understanding of what role all of this can play in driving marketing initiatives, we’re going to take a look at how three different companies are using data and combining that with testing to sell more merchandise.
Needless to stay, the case studies highlighted here or really any other one you find elsewhere should never be blindly copied – what works for one company is not guaranteed to work for another. There are simply too many variables to make an apples-to-apples comparison.
Instead, treat case studies as inspiration and use them to build out your own testing plans to see what works best for you.
How Appsumo used a simple survey to literally change a product overnight
A couple of years ago, Appsumo launched a new product called “How To Make Your First Dollar.” At launch day, hopes were high. With good reason. The product had been tested and well received by a test group. Plus, they had a highly targeted email list with 30,000 contacts.
What could possibly go wrong?
The official launch campaign attracted a measly 30 buyers. With a list of 30,000, that’s a conversion rate of just 0.1%. Something was clearly not right. To figure out what exactly happened, the decision was made to ask the people on the list directly via survey.
The brilliance in this wasn’t so much the action of launching a survey, but to whom the survey was sent. They targeted people who had showed some interest (opened the email, clicked through), but for whatever reason ended up not buying.
The survey had 4 simple questions:
- Were you at least interested in buying? YES or NO
- Be specific about the answer
- What’s holding you back from starting your business?
- Should we make our support sumo do a dance video?
From the responses, they deleted anyone who wasn’t interested in buying (based on answers to question #1), and the rest were sorted by response frequency.
Armed with this data, they went ahead and did a complete redesign of the landing basic using customer’s language and provided answers to the most burning questions. Now, Appsumo didn’t provide exact improvement numbers, but what they did say was that “This one survey literally changed the product overnight.” That’s good enough for me.
List segmentation leads to 377% increase in email revenue
Motorcycle Superstore is an online retailer specializing in all manner of moto gear like clothing, parts, tires, helmets, and more. They have used email marketing for years primarily to send offers and specialized content with the caveat that they send the same content to everyone – no segmentation whatsoever.
With the strategy of “sending everyone everything,” they managed to achieve average open rates of 18.5% and CTR of 6.2%. Not the worst in the world, but definitely not the best.
Luckily for them, the motorcycle industry is roughly divided into three categories: off-road, sport-bikes, and cruiser. That, plus additional data from purchase history, email click activity, and a post-purchase survey allowed them to divide the list into six segments:
- Combination of two riding styles (totaling 3 segments)
- Combination of all 3 styles
Next, they customized their email campaigns to match each segment.
That included a unique look and feel + different merchandise offers depending on the segment. All that lead to open rates doubling (18.5% -> 38.6%) and CTR more than tripling (6.2% -> 20.6%).
In addition to implementing proper segmentation, Motorcycle Superstore also started to A/B test various things regarding email starting with basics like time of the week, time of day, frequency, and subject lines. They moved into more complex things as they gained more experience.
All in all, those strategies lead to 377% increase in revenue generated through email. 377% increase in revenue! Not bad.
Email from field optimization leads to 137% higher open rate
Email marketing is one of the areas where data plays perhaps the most important role or at the very least, an area where getting and understanding the data being generated is relatively easy. Everyone can understand what a 20% open rate means.
Because of this easily accessible data, a lot of testing goes into perfecting email marketing campaigns from subject lines to timing to different offers etc.
One of the areas that doesn’t get a lot of attention, though, is the “from” field. It seems like such a small thing. What difference does it really make if my campaigns are under a generic company name, a real person, or a combination of both?
Turns out that it can make a big difference.
MarketingSherpa wanted to see if using an actual person would drive more opens and CTR, and so they ran a test with one version coming from “MarketingSherpa” and the other from “Jon Hosier, MarketingSherpa.” In both cases, the same subject line (“[Webinar] 4 steps to drive a measurable social strategy”) was used.
In the end, the more personal Jon Hosier version was able to achieve 137.4% higher open rate. Exact same subject line, just different name and a sizable improvement.
HubSpot saw similar results when they tested email from “HubSpot” or from “Maggie Georgieva, Hubspot.”
In their testing, the improvement in percentage terms was pretty low – opens went from 6.57% to 7.10% and CTR from 0.73% to 0.96% with the personal version. Still, even with those small gains HubSpot ended up with 292 more clicks which translated into 131 more leads from just one email campaign.
To get you started, here are some example “From” field names to test:
- Company – Klaviyo
- Company newsletter – Klaviyo Blog
- Company department – Klaviyo Support
- Team member name – John Doe
- Team member name and title – John Doe, CEO
- Team member name and company – John Doe (Klaviyo)
- Team member first name only – John
- Team member first name and company – John from Klaviyo
In online commerce there are endless opportunities for data collection and analysis, these are just three examples of how using the data at your disposal can lead to amazing results. Read through these case studies, study them, figure out how they relate to your business and iterate on the ideas brought forward.
Also, don’t be afraid to combine these ideas and come up with your own. After all, Appsumo didn’t just launch a random survey and hoped for the best. No. They sent the survey out to a segmented list that included only the people who could actually help them.
And finally, as is always the case with any case study, don’t just blindly copy what you see. What works for one company is not guaranteed to work for another. Instead, figure out how you can take these ideas and iterate on them to make them your own. Good luck.