Awesome A/B Tests for Flows

A/B tests

A/B testing flows is a little different than A/B testing newsletter campaigns. Because flows are a series of emails that run continuously in response to an action recipients take, they give you a more direct comparison between variations A and B. Rather than testing concepts, you can see how a variation in each email impacts opens, clicks, and conversions. As a refresher, here are the metrics you should look at in an email:

  • Open Rate — out of all the emails delivered, how many were opened
  • Click Rate — out of all the emails delivered, how many were clicked
  • Conversions — out of all the emails delivered, how many resulted in a purchase (or other action that you have set as your conversion metric)

And here are the components of an email that you can test:

  • Subject Lines
  • Preview Text
  • From Address
  • Content
  • Layout

Instead of using time as the benchmark for how long your A/B tests run, you should use number of emails delivered. Depending on the size of your business, this can range anywhere from 100-10,000 emails delivered.

Then, once you determine a winner, you can begin another A/B test — until you’ve reached an open, click, and conversion rate that you’re happy with.

You can also A/B test the number of emails you send by creating two separate flows. If you go this route, it’s important to make sure that when recipients trigger each series, they’re sent to either one or the other (not both!).

For example, you might have a 5-email series that you want to test against a series of 4 to see if 5 emails are too many. You can see if there are too many by monitoring the unsubscribe rate. Just make sure the emails in the series that you’re not testing are identical.

Let’s take a look at some ideas for testing your five main flows.Welcome SeriesSubject Lines & Preview Text
For your welcome series, pay special attention to testing name personalization and tones. Your welcome series is aimed at inviting new subscribers into your brand and connecting with them on a 1:1 level. It’s a great opportunity to get personal.

You also want to strike a balance between welcoming new subscribers into your world and persuading them to make a purchase. Here are some ideas to get you started:

  • Marissa, welcome to the club. Here’s 10% off your first purchase
  • Welcome, Marissa — the first one’s on us
  • Marissa, you’re in! Here’s a gift from us to you
  • Thanks for signing up, Marissa! We have something special for you

For all of these subject lines, you can easily test the effect of personalization by removing the recipient’s name.

Content & Layout
In addition to testing the design of your welcome emails, you should consider testing plain text versus HTML, since plain text emails can have a more personal feel. Instead of trying to sell your new subscriber right away, you could take this opportunity to personally welcome them into your community. By monitoring clicks and conversions, you might find that your audience responds better to plain text emails.Abandoned Cart & Browse AbandonmentSince abandoned cart and browse abandonment flows are very similar, many of the same tips are applicable to both.

Subject Lines & Preview Text
Abandoned cart and browse abandonment emails are great opportunities to play with a sense of urgency in your subject lines. Highlighting scarcity is an efficient way to get people who have already shown interest in one of your products to return to your site and follow through with the purchase.

You can also test using personalizing the subject line with the name of the product a shopper viewed or added to their cart. Here are a few subject line ideas to start with:

  • You forgot your [X product name]
  • Your [X product name] is waiting for you!
  • Hurry, before they’re sold out!
  • Only X of these left, hurry!

Content & Layout
There are a couple different angles you can take with your abandoned cart and browse abandonment emails: product-focused or brand-focused. Each will have a different layout and copy that speaks to its focus. Product-focused emails center around the item that a shopper left in their cart or recently viewed. As such, they should feature a large image of the item and a sense of urgency to draw shoppers back to your site.

Brand-focused emails, on the other hand, are primarily geared at providing shoppers with the help they need to purchase or alleviating any concerns that might be holding them back. They should feature help resources (think: sizing guides and return policies) and an easy way to get in touch with your sales or support team.

You can also experiment with including product feeds or recommendations in your abandoned cart and browse abandonment emails. While you don’t want to distract from the viewed or abandoned item, you might find that providing shoppers with alternative options works well for your business — the only way to know for sure is to test.Post-PurchaseThere are several different types of post-purchase emails, from simple “thank you” notes to product reviews to cross-sell emails.

When testing your post-purchase emails, you might want to go the route of testing different flows that we mentioned earlier to measure the impact of timing and the number of emails you send on opens, clicks, conversions, and unsubscribe rates.

Subject Lines & Preview Text
You can try personalizing your subject lines by including the name of the product that the recipient purchased. This works especially well for product review emails, since it allows you to use subject lines like:

  • How do you like your [X product]?
  • Please review your [X product]
  • Do you have anything you’d like to let us know about your experience with [x product]?

You can also try using the purchased product in your cross-sell subject lines. For example, you could use something like, “A lot of customers who liked [X product] also liked these!”

You can also use the recipient’s name in the subject line for your “thank you” post-purchase emails to make them feel friendlier and more personal.

Content & Layout
In addition to testing the design of your post-purchase flow emails, you might also consider testing plain text against HTML. As we mentioned earlier, plain text emails convey a more direct, personal tone. You might find that your audience responds well to a heartfelt “thank you” after making a purchase.

When it comes to product recommendation emails, you can test dynamic product feeds against a hand-picked selection of products. You can also test the number of product recommendations you include in the email — for example, three versus six. Win-BackLike your post-purchase flow, it’s a good idea to A/B test when you send your win-back flow and the number of emails in the series to see how your audience responds. Winning back inactive customers can be tricky, so it’s important that you test various options so you can get it right.

Subject Lines & Preview Text
You can test personalizing your subject lines based on the recipient’s first name, but this is also a good opportunity to be funny. Since you’re trying to re-engage customers (or, more literally, win them back), you can adopt a cheekier tone than you usually might. For example:

  • Baby, come back
  • Where did we go wrong?
  • I miss you
  • Give me one more chance

You can also test varying discounts in your welcome series, which you should call out in your subject line. For example, you might find that 10% off actually is more persuasive than 15% off.

Content & Layout
Testing discounts has to be reflected in both the subject line and the body of the email, so this is one exception to the “only change one variable at a time” rule.

Try testing a clear, hyper-focused CTA against product feeds or recommendations to see which drive more conversions. While streamlined CTAs are usually the way to go, for Win-Back flows you may want to provide lapsed customers with a variety of options that they can choose from. Bottom LineWhen A/B testing flows, first set a threshold for the number of emails you should test. This will, of course, depend on the size of your business. The great thing about flows is that once you’ve reached this threshold and arrived at a winner, you can switch out the losing variation for a new one and continue to test.


Blank Label Drives Showroom Bookings With Klaviyo



Klaviyo Launches Intelligent Custom Groups to Drive Better Customer Ou…



Copywriting in Email Templates


.yuzo_related_post img{width:260px !important; height:250px !important;}
.yuzo_related_post .relatedthumb{line-height:16px;background: !important;color:!important;}
.yuzo_related_post .relatedthumb:hover{background:#ffffff !important; -webkit-transition: background 0.2s linear; -moz-transition: background 0.2s linear; -o-transition: background 0.2s linear; transition: background 0.2s linear;;color:!important;}
.yuzo_related_post .relatedthumb a{color:#323b43!important;}
.yuzo_related_post .relatedthumb a:hover{ color:}!important;}
.yuzo_related_post .relatedthumb:hover a{ color:!important;}
.yuzo_related_post .yuzo_text {color:!important;}
.yuzo_related_post .relatedthumb:hover .yuzo_text {color:!important;}
.yuzo_related_post .relatedthumb{ margin: 0px 0px 0px 0px; padding: 5px 5px 5px 5px; }

jQuery(document).ready(function( $ ){
//jQuery(‘.yuzo_related_post’).equalizer({ overflow : ‘relatedthumb’ });
jQuery(‘.yuzo_related_post .yuzo_wraps’).equalizer({ columns : ‘> div’ });

Back to Blog Home
Get email marketing insights delivered straight to your inbox.