How can I create an A/B split test campaign?

A/B Split test campaigns are the best thing ever. The idea is simple: you want to check out the impact of one particular change in your subscribers, how they like it, how they respond to it. This answers the age old email marketing question: what if I did it in a different way?

With Moosend's A/B split testing you'll be improving your campaigns, testing different versions of your designs and getting to know what your subscribers prefer, what they go for and what works perfect and gets the results you want.

The process is super simple, just follow the steps below:

A. Locating the A/B Split testing

1. Click on New Campaign button, located on the top of your Dashboard page. ab-split-test-campaign-A1.png

2. Click on the A/B Split Test Campaign tab, to define the type of the campaign as such. ab-split-test-campaign-A2.png

3. Choose upon which characteristic of the campaign you'll be basing your test. ab-split-test-campaign-A3.png

You have the following options to choose from:

  • Create different versions of the Subject Line
  • Create different versions of the Campaign Content
  • Create different version of the Campaign Sender

Keep in mind that from a statistical point of view it makes no sense to check two characteristics at the same time - you wouldn't be able to know which is the one that had the desired effect! That's why our platform allows gives you the option to make an A/B Split test based on one characteristic at a time. 

Let's check each option in more detail.

B. The different A/B Split test characteristics

a) Different Subject Lines

Deciding the proper title for your campaign means you'll be getting a better open rate! By setting a Subject A and a Subject B for your campaign, testing the two on a percentage of your list and then automatically sending the final campaign with the subject line that performs better to the rest of your recipients you'll be sure the increase your numbers, not just this time, but every time 83-AB-SPLIT-CAMPAIGN-FACTORS-A.png  

b) Different Content

Τhe second option allows you to check two different HTML files, which could have different messages, different text, different images etc. 

As usual you'll have to set an easily identifiable Campaign Name for your own usage, a Subject line and a Sender for your campaign. The two content versions will be added on a later step: when you reach the point of having to define the HTML Content A and Content B you'll be prompted to import two different HTML files, either from a web location or directly from your computer.

Depending on how well the two versions perform on a test group, the rest of your recipients will be receiving the best HTML content, which is the one the test group interacted with the most. 83-AB-SPLIT-CAMPAIGN-FACTORS-B.png  

c) Different Senders

It's not always an easy pick between using a person's name as the Sender and making your campaign look more friendly, or opting for your company's name to make it sound more professional. So let your recipients decide which one they like best! 83-AB-SPLIT-CAMPAIGN-FACTORS-C.png

C. Creating the campaign

Once you've settled upon which available A/B Split characteristic you want to test, you'll then proceed with the standard campaign steps.

1. Click on the Next button. ab-split-test-campaign-C1.png

2. Tick the Mailing Lists which you want to use as targets for the campaign. ab-split-test-campaign-C2.png

3. Choose whether you want to use Entire lists or Segments of lists. ab-split-test-campaign-C3.png

You can add and combine as many Mailing Lists or List Segments as you want, there is no limit.

4. Click on the Next Button. 

5. Add your campaign content. ab-split-test-campaign-C5.png

You can either Import your campaign using an HTML file or link to your campaign content using a URL. Both options are available on the Import Campaign tab. Alternatively you can create customized results by using Moosend's super powerful Campaign Editor.

6. Click on the Use Campaign Editor tab, if you prefer to use our Editor. ab-split-test-campaign-C6.png

It is important to remember that if your A/B Split Test Campaign is based around two different versions of content, you will then have to define the content for the test twice, once for each version!

7. Click the Next button, to define the winning version of the A/B Split test.

D. Setting up the A/B Split test parameters

You will now have to define which part of your mailing list will act as a test group and which one as a winner group. As you may have already guessed, the test group will receive the different campaign versions and based on how they react the winning version will be defined and sent to the rest of the mailing list, which is called the winner group.

1. Click-n'-drag the white bar, to set the number of subscribers in the test group. ab-split-test-campaign-D1.png

The yellow segment of the bar represents the amount of people who will be receiving the winning version of your campaign. Which subscribers falls in which group is completely random and even if you did the same test twice the people in each group would never be exactly the same!

2. Choose the rate by which the winning version is decided. ab-split-test-campaign-D2.png

You have two options:

  • The campaign version with the highest number of unique opens wins
  • The campaign version with the highest number of unique clicks wins 

3. Click-n-drag the white bar, to set the duration of the test before a winning version is decided. ab-split-test-campaign-D3.png

The maximum duration for the test is 24 hours, whereas the minimum is 1 hour.

4. Click on the Next button when you're ready to proceed. ab-split-test-campaign-D4.png

E. Dispatching the campaign

The rest of the procedure is similar to any other campaign creation procedure you followed in order to send out a campaign. All that remains to be done is that you set the parameters so that your campaign is dispatched appropriately.

1. Click on the Skip Test button. ab-split-test-campaign-E1.png

You can always send a test mail if you so wish, but bear in mind that the test email address will only be receiving the A version of the A/B Split Test Campaign.

2. Choose the Send this campaign immediately option if you want your campaign to be dispatched right away. ab-split-test-campaign-E2.png

Your campaign will be placed on our delivery queue and will be sent out right away.

3. Choose the Schedule this campaign to be sent out at a future date and time option if you prefer to have your campaign be sent out on a date and time that you specify. ab-split-test-campaign-E3.png

4. Click on the Date and Time field to set the desired date and time for your campaign launch, using the pop-up window that appears. ab-split-test-campaign-E4.png

5. Click on the Done button. ab-split-test-campaign-E5.png

6. Click on the Timezone drop-down menu. ab-split-test-campaign-E6.png

Be extra mindful of the relation between your time zone, as you've set it on your Account Settings and the time of delivery of your campaign. Here's a simple example: if you have chosen to send your campaign at 16:00 and you have set your timezone is GMT+1, then your campaign will be sent out at 17:00.

7. Click on the Next button, to apply your changes. ab-split-test-campaign-E7.png

The Snapshot is the last step of the process and serves as an overview, where you can see all the settings of your A/B Split Test Campaign before dispatching it. As soon as you're ready, click the Send button and your campaign will be on its way!