Crear cuenta Mailrelay Crear cuenta Mailrelay Crear cuenta Mailrelay

New feature for your emailings A/B Testing

Published by Mailrelay on March 4, 2014

We are continuously working so as to offer new features for our Mail relay users. We have recently posted an article about the option to get statistics for your campaigns sent using our SMTP server, as well as the new advanced filters for subscribers and of course, the feature to send automatic campaigns using a RSS feed.

Another feature that many of our customers were asking for is the option to use A/B testing, and that is what we bring you today.

An easy to use functionality that we are sure you all will love.

How can A/B testing help you with your marketing campaigns?

If you haven’t heard of this concept before, it is important to understand that A/B testing is one of the best tools that can help you to improve your emailings.

Let’s start with an example.

Imagine that you have 100 subscribers and you are about to start your new email marketing campaign.

You already created your newsletter, but you don’t know which subject, which title would be the best option.

It is a very important detail that can help you to improve your open rates.

It would be great to try two variations and see which one works best, wouldn’t it?

That is what we will do, of the 100 subscribers we will take a subgroup, for instance with 20 email addresses.

Of these 20 subscribers, you will send the newsletter with one subject to 10 subscribers, and to the other 10 the same newsletter with a different subject.

After sending the test, you can analyze the results, as you will see the percentage of open rates of each newsletters, you will be able to send the email with the subject you prefer to the remaining 80 subscribers.

Thanks to this, we can expect a better result in our emailing.

We will see now how to use this feature in the Mailrelay interface

The first thing you need is the newsletter base, so what you have to do is to create your newsletter as usual.

After you have created it, you cannot send the message, instead you have to click on the Test A/B menu:

test AB en 1

You will see  a screen similar to the following:

test AB en 2

Here we can see a brief explanation of the process and use of A/B testing as well as a list of previous tests.

The first step is to click on the Create button. After that you will see the following screen:

test AB en 3

The options presented are:

1.- Select the newsletter base, the original email on which you will work.

2.- The percentage of subscribers who will be used for the subgroup on which you will run the test will be runned.

3.- Finally, the type of test you want to run, you can test the subject line or test using a different sender name.

A sample configuration might be:

test AB en 4

In this case we have selected the newsletter A/B testing.

We have entered a percentage of 10% for the test.

After that we have selected the type of test as Subject, and the systemshowed us a field to add a different subject line.

How does it work?

This test will be sent to 10% of the subscribers belonging to this group. 5% of them will receive the email with the original subject and the other 5% will receive the newsletter with a different subject line.

To send the test, we just have to click on the Save button. This not only saves the test, but will send it:

test AB en 5

If you want to start the test immediately, click the Yes button.

IMPORTANT: The percentage of subscribers must be enough for the test to be meaningful, select it carefully. The smaller the test, less statistical information you will have to know what to do.

Now we will go back to the list of tests

After sending the test, you can return to the main screen, where it is possible to see it:

test AB en 6

Select it and click on the View button to access the details of the test:

test AB en 7

In this screen you will find all the information you need to decide which subject line will be best to use for the final campaign.

We have the amount of emails sent, how many have already been processed, clicks and views.

There are two options to compare the versions of the campaign, you can find the information graphically at the top or numerically below.

Both options will give you the information you need to decide which version you will send to the remaining subscribers.

As explained before, the higher the percentage of the sample, the more effective the test will be, as you will have more data to decide what to do.

Now the only thing you have to do is to select the newsletter you want to send (click on it on the list) and click on the button send selected newsletter for the remaining subscribers, so that the campaign will be sent immediately:

test AB en 8

All campaigns, including your tests are available in the menu Sent Newsletters -> Sent Newsletters:

test AB en 9

Therefore you can also see the statistics of the campaigns by accessing this menu.

Advantages of this new feature

This new feature not only gives you the ability to test between various subject lines, but it gives you complete flexibility to launch the final campaign.

It is very important to allow enough time after running the test, because you will have more information that will help you to evaluate the results.

Most email marketing tools do not wait so long, and after getting the results sen the best option to the remaining subscribers.

However, in my opinion, it is better to give some time, it is possible thatmany users will only open the newsletter several hours after the campaign was sent.

We must also evaluate more aspects of our email marketing campaigns, perhaps the newsletter with the best open rates received a lower percentage of clicks.

You may prefer to send the version with fewer openings, but with more clicks.

You can decide when to send your campaign and which version will be sent, even if it was not the winning campaign.


Suscríbete al blog y recibe los mejores contenidos semana a semana:

Acepto la política de privacidad.


  1. Pol

    ¿Es posible programar el envío del ganador?

    PJ: 1 día después, 3 días después, X horas…

    Tener que activarlo manualmente…

    20/07/2017 - 15:10:57 Publicar una respuesta
  2. Miguel Abreu E.

    Gracias, lo probare ha ver que tal la herramienta.

    07/03/2014 - 16:48:46 Publicar una respuesta
  3. Andrés

    Voy a probarlo, me parece una herramienta muy útil.

    06/03/2014 - 15:49:59 Publicar una respuesta
  4. Shuberth Chi Balam

    Muy buena forma de aumentar la efectividad de los correos.

    At. Shuberth Chi Balam

    05/03/2014 - 16:11:09 Publicar una respuesta

Leave a comment

Your email address will not be published. Required fields are marked *

Menu Title