How can I create an A/B split test campaign?

Using our new UI? You might want to check this help article instead.


A/B Split Campaigns are the best thing ever. The idea is simple: you want to check out the impact of one particular change in your subscribers, how they like it, how they respond to it. This answers the age old email marketing question: what if I did it in a different way?

With Moosend's A/B split Campaign you'll be improving your campaigns, testing different versions of your designs and getting to know what your subscribers prefer, what they go for and what works perfect and gets the results you want.

The process is super simple, just follow the steps below:

A. Locating the A/B Split Campaign

1. Click on New Campaign button, located on the top of your Dashboard page. ab-split-test-step_A1.png

2. Click on the A/B Split Campaign , to define the type of the campaign as such. regular_step_1-_no_number.png 

3. Choose upon which characteristic of the campaign you'll be basing your test. 

You have the following options to choose from:

  • Create different versions of the Subject Line
  • Create different versions of the Campaign Content
  • Create different version of the Campaign Sender

Keep in mind that from a statistical point of view it makes no sense to check two characteristics at the same time - you wouldn't be able to know which is the one that had the desired effect! That's why our platform allows gives you the option to make an A/B Split Campaign based on one characteristic at a time. 

Let's check each option in more detail.

B. The different A/B Split Campaign characteristics

a) Different Subject Lines

Deciding the proper title for your campaign means you'll be getting a better open rate! By setting a Subject A and a Subject B for your campaign, testing the two on a percentage of your list and then automatically sending the final campaign with the subject line that performs better to the rest of your recipients you'll be sure the increase your numbers, not just this time, but every time ab_step_b1.png  

b) Different Content

Τhe second option allows you to check two different HTML files, which could have different messages, different text, different images etc. 

As usual you'll have to set an easily identifiable Campaign Name for your own usage, a Subject line and a Sender for your campaign. The two content versions will be added on a later step: when you reach the point of having to define the HTML Content A and Content B you'll be prompted to import two different HTML files, either from a web location or directly from your computer.

Depending on how well the two versions perform on a test group, the rest of your recipients will be receiving the best HTML content, which is the one the test group interacted with the most. ab_step_b2.png  

c) Different Senders

It's not always an easy pick between using a person's name as the Sender and making your campaign look more friendly, or opting for your company's name to make it sound more professional. So let your recipients decide which one they like best! ab_step_b3.png

C. Creating the campaign

Once you've settled upon which available A/B Split characteristic you want to test, you'll then proceed with the standard campaign steps.

1. Click on the Next button. ab_step_c1.png

2. Tick the Mailing Lists which you want to use as targets for the campaign. 

3. Choose whether you want to use Entire lists or Segments of lists. 
You can add and combine as many Mailing Lists or List Segments as you want, there is no limit.

4. Click on the Next Button. 

5. Create your template. ab_step_c5.png
You can either Import your campaign using an HTML file or link to your campaign content using a URL. Both options are available on the Import Campaign tab. Alternatively you can create customized results by using Moosend's super powerful Campaign Editor.

6. Click on the Use Campaign Editor tab, if you prefer to use our Editor. ab_step_c6.png

It is important to remember that if your A/B Split Campaign Campaign is based around two different versions of content, you will then have to define the content for the test twice, once for each version!

7. Click the Next button, to define the winning version of the A/B Split Campaign.

D. Setting up the A/B Split Campaign parameters

You will now have to define which part of your mailing list will act as a test group and which one as a winner group. As you may have already guessed, the test group will receive the different campaign versions and based on how they react the winning version will be defined and sent to the rest of the mailing list, which is called the winner group.

1. Click-n'-drag the white bar, to set the number of subscribers in the test group. ab_step_d1.png
The yellow segment of the bar represents the amount of people who will be receiving the winning version of your campaign. Which subscribers falls in which group is completely random and even if you did the same test twice the people in each group would never be exactly the same!

2. Choose the rate by which the winning version is decided. ab_step_d2.png

You have two options:

  • The campaign version with the highest number of unique opens wins
  • The campaign version with the highest number of unique clicks wins 

3. Click-n-drag the white bar, to set the duration of the test before a winning version is decided. ab_step_d3.png

The maximum duration for the test is 24 hours, whereas the minimum is 1 hour.

4. Click on the Next button when you're ready to proceed. ab_step_d4.png

E. Dispatching the campaign

The rest of the procedure is similar to any other campaign creation procedure you followed in order to send out a campaign. All that remains to be done is that you set the parameters so that your campaign is dispatched appropriately.

1. Click on the Skip Test button. ab_step_e1.png

You can always send a test mail if you so wish, but bear in mind that the test email address will only be receiving the A version of the A/B Split Campaign.

2. Choose the Send this campaign immediately option if you want your campaign to be dispatched right away. ab_step_e2.png

Your campaign will be placed on our delivery queue and will be sent out right away.

3. Choose the Schedule this campaign to be sent out at a future date and time option if you prefer to have your campaign be sent out on a date and time that you specify. ab_step_e3.png

4. Set-up the appropriate fields with the required information.ab_step_e4.png
You'll need to specify the following:

  • The date and time of delivery
  • Your time zone

Be extra mindful of the relation between your time zone, as you've set it on your Account Settings and the time of delivery of your campaign. Here's a simple example: if you have chosen to send your campaign at 16:00 and you have set your timezone is GMT+1, then your campaign will be sent out at 17:00.

5. Enable the Automated Resending if you want to schedule and send your campaign again to the subscribers who did not open it.ab_step_e5.png

6. Set-up the appropriate fields with the required information.

You'll need to specify the following:

  • The time that should elapse after the original campaign is delivered.
  • The subject of your resent campaign. You can leave this field empty if you do not wish to change the subject.
  • The sender of your resent campaign.

7. Click on the Next button when you're ready.

The Snapshot is the last step of the process and serves as an overview, where you can see all the settings of your A/B Split Campaign before dispatching it. As soon as you're ready, click the Send button and your campaign will be on its way!