Email A/B Testing: A Full Guide

If your impression of marketing is solely informed by episodes of Mad Men, you may think that artistic intuition and a strike of good luck is all you need to succeed. Despite Don Draper’s fictional achievements, modern marketers must be creative and analytical at the same time.

In fact, in today’s digital age, data is the driving force behind any modern marketing campaign. 

And one of the best methods for gathering that data is A/B testing—especially when it comes to email.  

This process has the power to boost your emails’ open rates, click-through rates, and conversion rates, ultimately garnering you more sales and revenue. In this guide, we’ll explain how you can implement an A/B testing email marketing strategy in future campaigns step by step.  

What is A/B Testing?

Put simply, A/B testing (also known as split testing) is the process of sending two versions of an email to a subset of your subscribers. 

Half of the subset receives email A, while the other half receives email B.

The two versions of your email A/B testing may look very similar, but they’ll contain one small variation, such as a different:

• Subject line
• Layout
• CTA
• Images
• Copy length
• Copy tone
• Testimonials
• Send time

Once your subscribers have had enough time to interact with these two emails, you can compare their open rate, click-through rate, or conversion rate (depending on what metric you’re trying to optimize). After identifying the better-performing email, you can send this version to the remainder of your subscriber list. 

The Power of A/B Testing

You may have a hunch about what will capture your target audience and win their conversions, but unless you test it out, you’ll never know with confidence. 

A/B testing is a surefire way to pinpoint what your audience responds well to. You may be surprised that they prefer different things than you originally hypothesized. 

Despite the simplicity of email A/B testing and its many benefits, 39% of brands aren’t using it yet.¹ Neglecting this powerful technique means that your email testing campaign may not be reaching its full potential. In turn, you could be missing out on a ton of business. By A/B testing your email campaigns going forward, you can capture more customers and gain an edge over your competition.

How to Run an A/B Email Test

If you’re ready to employ this game-changing technique, just follow these steps:

#1 Determine Your Sample Size

As with any scientific experiment, you need an adequate sample size to yield statistically significant results. 

Your subscriber list dictates your available sample size:

• Large subscriber list – If your subscriber list is extensive, you can test your two emails on a smaller subset of people first. For instance, you could send 10% of your subscriber list one version of the email and another 10% the other version. Once the results are in, you can send the better version to the remaining 80%. 
• Small subscriber list – If your email list is on the smaller side, you may want to conduct A/B tests on your entire audience, splitting the recipients 50/50. This will ensure your results are representative of your entire audience. 

#2 Set Up Your Chosen Software

Many email software offer built-in A/B testing capabilities, such as Mail Chimp, Constant Contact, Campaign Monitor, and Active Campaign.

Their A/B testing software allows you to send a different version of your email to two subsets of your subscribers. Once the results are in, the winning version of the email will be sent to the rest of your subscriber list automatically. 

If you don’t use these email software tools, you can run your A/B test manually. All you have to do is incorporate email segmentation within your subscriber list into two sections and send a different version of the email to each one. After that, you’ll have to analyze their performance manually.  

#3 Select Your Variables

Next, it’s time to brainstorm what specific changes may improve your email’s performance and email accessibility. Maybe you suspect that the subject line could be snappier. Or perhaps you think some personalization could inspire more conversions. 

Choose a change you think has the most potential to improve your campaign and add it to the B version of your original email. 

Here are a few aspects of your email you can run A/B tests on:

Subject line

Your email’s subject line can make or break your open rate. If you can’t get subscribers to open your email, all of your hard work on its design and copy will go to waste. 

Here are a few tweaks you can implement during subject line testing to see how they impact your email’s open rate:

• Rearrange the order of the words within your subject line
• Phrase the subject line as a question
• Include numbers or emojis
• Play around with capitalization
• Test out different subject line lengths
• Use language that instills a greater sense of urgency
• Add more personalization

According to research, subject line personalization is one of the most powerful tools you can use to improve your email’s open rate. It’s been shown to boost open rates by up to 14%!²

Preheader Text

Another factor that can impact your email’s open rate is its preheader text. This text shows up as a continuation of your subject line within your readers’ inbox. 

In other words, preheader text gives you even more space to capture readers’ attention and inspire them to click. 

Sender’s Name

One final factor that may impact your email’s open rate is the sender’s name. This is the email address you send your email from that shows up in your recipients’ “From” field. 

During an A/B test, you could send one email from a real person at your company (john@company.com), rather than the company’s name (info@company.com). 

Forging this kind of personal connection from the get-go may be what you need to get more readers to click. 

Call to Action (CTA)

Once a subscriber has opened your email, the next goal is to get them to click on your CTA. For this reason, your CTA is one of the most important features to A/B test.

You can test your CTA’s:

• Copy
• Font
• Text size
• Capitalization
• Button color
• Location
• Frequency within the email 

You’ll know your CTA changes are working if they result in more clicks. 

Visuals

When it comes to your email’s design, your audience’s preferences may be different than your own. Some subscribers may love a lot of images, while others may prefer plain text. 

Find out what your audience likes by experimenting with different visuals. You can A/B test:

• Images vs plain text
• A single image vs. multiple images
• Drawings vs. real-life photos
• Animated GIFs vs. still images
• Colorful images vs. monochrome images
• Stock images vs. company photos
• Professional images vs. relatable memes

While it may take several split tests to narrow down exactly what works best with your audience, this degree of precision can help you design highly optimized email campaigns that resonate with them to a tee. 

Length and Tone of Copy 

Most people have very short attention spans, especially when it comes to reading marketing materials. Microsoft discovered that the average person’s attention span has fallen from 12 seconds in 2000 to a mere 8 seconds in 2021.³ For reference, goldfish have attention spans of 9 seconds. 

The jury is still out on whether TikTok or Instagram is to blame for this attention span deficit. What we do know is that your email’s copy has to grab readers’ attention right away. Otherwise, you’ll lose out on a ton of potential clicks.

You can boost your copy’s effectiveness by A/B testing its:

• Headers – The first line of copy in your email must capture readers’ attention or they won’t spend a second longer scanning the page.
  
• Length – Shorter copy may bode better for audiences with short attention spans. Make sure every word in your email highlights the value of your offer and guides readers to your CTA.
• Tone – The tone of your copy needs to resonate with your target audience to get them engaged. Whether your brand is friendly or authoritative, studies have shown that using positive language improves email engagement.⁴ 

Email A/B Testing: Blending the Art and Science of Email Marketing

As you can see, A/B testing is a straightforward process—it’s also an ongoing and time-consuming one. We’ve only outlined a few of the factors you can put to the test with this process. You can also A/B test different send times, testimonials, layouts, and more. 

As you compile more data about your audience’s preferences, you’ll become an expert at delivering them emails they love. And you’ll know statistically what yields the most success. In that sense, you can consider yourself a data-driven Don Draper of the 21st century. 

If you’re looking for more support to maximize the effectiveness of your email campaign—or your great marketing strategy—we can help. Looking for further assistance with SMS compliance or SMS marketing? Our digital marketing agency has the expertise and tools needed to optimize your marketing efforts and expand your reach.  

Sources:

1. Litmus. 13 Email A/B Testing Mistakes that Limit Your Success. https://www.litmus.com/blog/email-a-b-testing-mistakes-that-limit-your-success/
2. Campaign Monitor. 70 Email Marketing Stats Every Marketer Should Know. https://www.campaignmonitor.com/blog/email-marketing/70-email-marketing-stats-you-need-to-know/
3. The Telegraph. Humans Have Shorter Attention Span than Goldfish.  https://www.telegraph.co.uk/science/2016/03/12/humans-have-shorter-attention-span-than-goldfish-thanks-to-smart/
4. Campaign Monitor. What Is a Good Email Click-Through Rate for 2019? https://www.campaignmonitor.com/blog/email-marketing/what-is-a-good-email-click-through-rate-for-2019/