A/B vs Multivariate – What’s the best test?
The quest for higher page conversion is a never ending journey. Simpler forms, better images, different call to actions, different coloured buttons, long pages, short pages - so many tests, so little time. So what’s the best way to approach the tests you should be running to help with your quest for the holy grail of conversion. Here’s a couple of ways you might want to tackle testing.
A/B Testing
Probably the most common and technique used in optimising page conversion is to run A/B testing on the section of your site you want to test. The ideal scenario for A/B testing is when you want to test two completely different designs against one another or if only one element of the page is varied.
Besides its simplicity, one other advantage of A/B testing is that even sites with lower traffic volumes can quickly obtain meaningful results given traffic is only spread between one or two different pages.
Besides its simplicity, one other advantage of A/B testing is that even sites with lower traffic volumes can quickly obtain meaningful results given traffic is only spread between one or two different pages.
Multivariate Testing
Complementing A/B testing technique is the lesser used Multivariate testing technique. As the name suggests, this technique tests ‘multiple’ ‘variations’ against each other rather than the smaller numbers of versions in A/B testing.
The major difference in multivariate testing is in its use. Rather than just testing different page versions, multivariate is designed for testing different elements inside a given page. For example – a landing page may contain a headline, some written content, a form, and a button. If there were 2 variations of each of these then we’d be looking at more variations than A/B testing could comfortably handle. Here’s where multivariate testing becomes the best fit.
But more versions are not always better and testing this way does have its weaknesses. Given the large number of pages that can be generated out of a relatively small amount of variables, the unsuspecting tester may find themselves with their website traffic split across 10-20 different versions of a particular page. If the website traffic was not high enough to produce a full range of meaningful data then it could potentially be weeks or months before enough data was collected to make any real decisions from the data collected.
So whether A/B or multivariate, the best way to desicde on which type of testing to use is to first decide what it is you want to test, design some alternate pages based on what you want to find out and then choose a testing framework that best fits both your website traffic volume and testing hypotheses.
