How to master A/B testing: nail the basics, then put optimization on autopilot
For years, marketers have used A/B testing to optimize marketing assets.
In 2012, Bing performed a small A/B test on the way their search engine displayed ad headlines. It led to a 12% increase in revenue—more than $100M in the US alone.
But that was over a decade ago. Since then, getting attention online has become a lot more competitive. Noah Kagan, founder of web app deals website Appsumo, recently revealed that only 1 in 8 of their A/B tests produces significant results—meaning brands now have to perform these tests 1) at scale and 2) beyond email to keep iterating at a competitive rate.
This is where advanced A/B testing can help.
A/B testing (or split testing) is a research methodology that involves testing variations of a marketing email, SMS message, landing page, or website form to determine which one yields higher open rates, click rates, conversion rates, and more.
Now, new A/B testing technologies, including AI, empower brands to strip away some of the manual work that used to come with A/B testing—and set steps on autopilot to start seeing results faster.
Here, we’ll get into some of the basics of A/B testing, but we’ll also talk about some of the ways your brand can use new methodologies to speed up those tests—and ultimately get more out of them.
What do you test in A/B testing?
Important note: When you’re performing an A/B test, never test more than one variable at a time. Doing so will yield an inconclusive result, because you won’t know which variable influenced performance.
Email A/B testing: what to test
Whether you’re A/B testing email campaigns or A/B testing email automations, here are a few elements to consider testing:
- “From” name (sender’s name that appears in an email’s inbox preview): your organization’s name vs. a more casual name (such as the first name of one of your team members) vs. a specific department of your organization, etc.
- Subject line: length, format, personalization elements, emojis, etc.
- Media: visuals vs. text, different hero images, gifs vs. static images, videos vs. infographics, etc.
- Email design and layout: single column, multi-column, image grids, F-pattern, zig-zag, different email templates, etc.
- Email content: offer and discount types, different user-generated content, voice and tone, length, emojis, etc.
- Calls to action (CTAs): number, placement, copy, color, buttons vs. hyperlinks, etc.
- Send time: day of week, time of day, etc.
How the right tech can make this easier—and make you more money
With a platform like Klaviyo, you don’t have to manually switch to sending the most successful variation of your A/B test.
Instead, you set the success criteria (more on this later) and/or the test duration, after which Klaviyo declares the winner automatically. The rest of the recipients will automatically receive the winning email.