Create an A/B split test campaign

Moosend lets you create A/B split campaigns to improve your campaigns, test different versions of your designs, and get to know what your subscribers prefer. You can create two different versions of the same campaign, send them to equal parts of your subscribers, and then automatically send the version which performed better in a set amount of time, to the rest of your list. This campaign also lets you test the subject line, campaign content, or campaign senders.

You can set all the information regarding your A/B split campaign through the campaign creation process described below.

A. Select the A/B split campaign type

  1. In the top navigation bar, click the Campaigns tab.
  2. On the Campaigns page, hover your mouse over New, and click A/B testing.create_RSS_campaign_2.png

B. Define the campaign settings

  1. In the Set basic settings step, select what you want to base your test on. create_AB_split_campaign_3.png
  2. You have the following options. It is recommended to make an A/B Split Campaign based on one characteristic at a time. 
    • Create different versions of the Subject Line
      You can get a better open rate with the proper subject line for your campaign. By setting a Subject A and a Subject B for your campaign, testing the two on a percentage of your list, and then automatically sending the final campaign with the subject line that performs better to the rest of your recipients, you make sure that you increase your numbers every time.create_AB_split_campaign_4.png
    • Create different versions of the content
      You can check two different HTML files, which could have different messages, different text, different images. You could also choose the Plain Text or HTML option. It is recommended to use the HTML format so that you can get accurate reporting out of your campaign which is not an option when using plain text.

      As with any campaign, you must set an easily identifiable campaign name, a subject line, and a "From" name.
      The two content versions are added on a later step: when you reach the point of having to define the HTML Content A and Content B you'll be prompted to import two different HTML files, either from a web location or directly from your computer.

      Depending on how well the two versions perform on a test group, the rest of your recipients  receive the best HTML content, which is the one the test group interacted with the most.create_AB_split_campaign_5.png

    • Create different versions of the "From" name
      You can select between using a person's name as the "From" name and making your campaign look more friendly, or opting for your company's name to make it sound more professional. create_AB_split_campaign_6.png


  3. When you have selected the A/B Split characteristic you want to test, proceed with the standard campaign steps by clicking Next.create_AB_split_campaign_8.png
  4. In the Choose email list step, select the email lists you want to use as targets for the campaign.

    Notes: You must select an email list to receive your campaign. You can choose more than one list, out of the email lists you have already created. You can also select any previously created segments of a list instead of an entire list, or any combination between different lists and segments. 

    Only the Active members of the selected email list receive your campaign. Any Bounced, Archived or Unsubscribed members are ignored and won't receive your campaign.

  5. If you would like to use a segment, in the Segments column, click the corresponding number and then choose your segment by selecting the check-box next to its name.
  6. create_AB_split_campaign_10.png
    Note: You can create a new segment if you haven't added any segments to your email lists.
  7. Click Next to continue the creation process.create_AB_split_campaign_11.png

D. Create your campaign design

  1. In the Design step, toggle the Imported campaign template switch to the On state in order to start the process.create_AB_split_campaign_12.pngYou can either Import your campaign using an HTML file or link to your campaign content using a URL. Both options are available on the Import CampaignTemplate tab. Alternatively, you can create customized results by using our platform's Campaign Editor.
  2. Click Start designing if you prefer to use the Campaign Editor.create_AB_split_campaign_13.png

    Important: Your A/B split campaign is based on two different versions of content, you will then have to define the content for the test twice, once for each version.
  3. Click Next to define the winning version of the A/B Split Campaign.create_AB_split_campaign_14.png


E. Set up the A/B Split Campaign parameters

You must define which part of your email list will act as a test group and which one is a winner group. The test group receives the different campaign versions and based on how they react, the winning version is defined and sent to the rest of the email list, which is called the winner group.

  1. In the Set AB Settings step, click and drag the white bar, to set the number of subscribers in the test group.
    Note: The white segment of the bar represents the number of people receiving the winning version of your campaign. Which subscribers fall in which group is completely random and even if you did the same test twice, the people in each group would never be exactly the same.create_AB_split_campaign_15.png
  2. Choose the criteria by which the winning version is decided.create_AB_split_campaign_16.pngYou have the following options:
  3. Click and drag the white bar, to set the duration of the test before a winning version is decided. The maximum duration for the test is 24 hours, whereas the minimum is 1 hour.
    The winning version is determined based on a campaign's performance for the test duration you have set on this step. For example, if you set the A/B test to run for one hour and after an hour, Campaign A has better results than Campaign B, then it becomes the winning version. Any campaign statistics received after that timeframe is ignored. So even if Campaign B performs better after two hours, these results are ignored because they are outside the test timeframe. 

  4. Click Next to proceed.create_AB_split_campaign_18.png

E. Dispatch the campaign

You can set the parameters so that your campaign is dispatched appropriately.

To schedule campaign delivery:

  1. In the Schedule Delivery step, toggle the I want to send this campaign now switch if you want your campaign to be sent right away. Please note that you will be able to preview your campaign settings in the next step before sending it.create_AB_split_campaign_19.png
  2. If you prefer to send the campaign at a later time, toggle the I want to schedule this campaign to be sent at a specific date and time switch. You can send it out on a date and time that you specify.create_AB_split_campaign_20.png
  3. Set up the appropriate fields with the required information. You need to specify the Date and time  and Timezone.create_AB_split_campaign_21.png
    Important: The data and time, and timezone you select here are used for the delivery of your campaign. This is different from what you have set up in your Account Settings
  4. Toggle the Resend campaign to non-openers switch to On state if you want to schedule and send your campaign again to the subscribers who did not open it.
  5. Fill out the fields with the required information.create_AB_split_campaign_23.pngYou must specify the following:
    • Resend campaign after -The time that should elapse after the original campaign is delivered.
    • Resend campaign subject line - The subject of your recent campaign. You can leave this field empty if you do not wish to change the subject.
    • Resend campaign "from" name - The sender of your resent campaign.
  6. Click Next to continue.create_AB_split_campaign_24.png


F. Preview your campaign

The Preview step is the last step of the process and serves as an area where you can see all the settings of your A/B Split Campaign before dispatching it. As soon as you are ready, click the Send button and your campaign will be sent to your subscribers. 

  1. In the Preview step, edit a setting by clicking the Edit link.
  2. Check for any missing information. Any fields with missing information are indicated by Missing_fields_icon.png.
  3. When you are ready to send your campaign, click Send. Otherwise, click Save & exit to save your campaign as a draft.

See AB testing campaign reports to learn more about reports generated in A/B split test campaigns.