Multivariate Testing

Test different combinations of web page 'variants' against each other. Multiple variants and possible combinations = more potential for conversion rate improvements.

Getting lots of visitors and want to run tests to find the optimum web page layout? Ready to make big improvements to your website's conversion rate? Multivariate testing is for you.

How Multivariate Testing Works

How do you go about improving the conversion rate of your web pages? Well first, let's break a page down a little further...

Page Elements and Variants

Most web pages will probably have at least some of the following:

• Buttons and links
• Headline
• Subheadings
• Body copy
• Forms
• Sign ups
• Images

These are your page 'elements' and they could all be improved (which would lead to a higher page conversion rate).

Multivariate testing is about running experiments to improve the performance of these elements. You are therefore going to need some test variants (of your page element).

Here are some possible variants of a website button:

Multivariate testing allows you to test multiple elements (through a series of variants) at the same time.
i.e. 'buttons + images' or 'images + headlines' or even 'buttons + images + headlines'

Complex Experimenting Made Simple(ish)

Due to the number of possibilities involved (multiple elements and variants) Multivariate testing has the potential to get complicated pretty quickly.

This is where Google Website Optimizer comes in - you can use this awesome tool to run and manage seriously complex experiments which would otherwise leave you and your team befuddled.

A little example might help...

Say you want to experiment with 2 page elements, image + headline, for example.

In a simple experiment you may want to test 2 variants of each element so you have 6 variants in total (2 control and 4 test variants)

Here's how your experiment would look in a visual matrix...

As you can see testing 2 elements with 3 variants of each (1 control and 2 test) means there are actually 9 possible combinations.

Once your experiment is set up Google Website Optimizer randomly shows each visitor 1 of the possible combinations (they will only ever see 1 version from the same computer). You can also set levels of traffic that see each different test combination.

Once you have had enough visits to your site Multivariate testing will tell you which combination of variants performs best.

Because of the number of page versions (or variant combos) involved successful multivariate experimenting requires good levels of traffic. It can also take a little time to reach 'conclusive results'.

To Conclude...

Use Multivariate Testing if:

• You are serious about improving your goal conversion rates
• You receive a good level of visitors to your site
• You believe in statistics and scientific methodology as the best way of improving your site
• You want to see big improvements in your site's performance

There is a great deal of science and maths that goes into establishing which combination of variants is the best. We won't go into it all now but feel free to get in touch and we will be more than happy to discuss what is meant by 'statistically significant' and how it might work for your site.