I know, I know! You’ve been asking for a way to do split testing for some time! Finally we have the first stage of our solution in place.
What It Is
You can set a template to go out to all orders or to odd numbered orders or even numbered orders. Amazon pretty much splits these order numbers 50/50 and we’ve done a lot of testing – this does seem to be a good way to do the splitting. By sending one template to odd numbered orders and another to even numbered orders you can see which is more effective.
Split testing (also referred to as A/B testing or multivariate testing) is a method of conducting controlled, randomized experiments with the goal of improving a website metric, such as clicks, form completions, or purchases. Incoming traffic to the website is distributed between the original (control) and the different variations without any of the visitors knowing that they are part of an experiment. The tester waits for a statistically significant difference in behavior to emerge. The results from each variation are compared to determine which version showed the greatest improvement.
How To Use It
What we recommend is to set up two almost identical templates (you can duplicate a template by using the ‘Clone’ button when you’re viewing a template). Set one of the templates to be an ‘A’ template and the other to be a ‘B’ template.
Then change something like the subject line. Changing the subject line in one template and leaving the other alone will, then looking at the open rates, will let you see which subject line is more affective.
Or leave the subject line and change the call to action in the content of the email.
Change only 1 thing between emails – otherwise you don’t know what caused the difference results!
Where to set it
The ‘A/B Selector’ is available while editing a template.
