A/B testing means testing two different versions of a webpage to see which works better. We compare the two versions of the website by showing visitors both versions (version A and version B) at the same time – some visitors get to see variation A, others variation B. After statistical significance or some other learning point is achieved we stop the test and use the insight for further improvements of the page.
All pages on the Internet have a goal – the reason for their existence, what they want to achieve.
Sales websites want visitors to buy / order products
News and media sites want visitors to click on ads or sign up for paid content
Every company that operates on the Internet wants as many visitors as possible. But at the end of the day, we want all visitors to become our customers. And the metric that tells us how many people are converting from a visitor to a customer / subscriber is the conversion rate. Measuring the performance of versions A and B tells us how many visitors turned into customers / subscribers – that would be a typical goal for which we optimize.
There is more to conversion rates than just a number. We have written extensively about it in our blog post Is the Marketing Funnel Really Dead?
Why would we even A/B test a website?
With A/B testing, we want to determine which version of a website is more successful in sales. The figure shows an example where the A page variation is more successful than the B page variation.
A/B testing allows you to make visitors to a website more than just visitors. While the price for a paid visit is relatively high, the price for a conversion increase is minimal.
What you can test
It makes sense to test almost all elements on the web as part of A / B testing. These include:
- What colors should the titles be?
- How should titles be written, how bold, and what lengths?
- Where should the text be placed?
- How should the text be formatted?
- What words should be used in the text?
- Where should the action-oriented buttons be placed?
- What color should the buttons be?
- What sizes should the buttons be?
- Which text is more appropriate for a call to action?
- What size should the whole module be?
- What is the better layout on the website itself?
- Price lists,
- Price structure,
- Sales promotion items,
- Free postage.
A / B testing process
To make the A / B testing process as effective as possible, it is crucial that you follow all the fulcrums that guide you through the research and analysis of the data you get through testing.
- Start with a question. “Why is the bounce rate on my website higher than on other websites in the same industry?”
- Background research. You need to understand the behavior of your visitors, which is made possible by advanced Google Analytics analysis. The analysis allows you to find out how visitors to the website behave – where they come from, how long they are on the website, where they leave the website, which subpages they look at…
- Have a hypothesis. For example: “If we add more links to the foot of the website (to other subpages) we will reduce the bounce rate on the home page.”
- Calculate how many visitors you need per day to start A / B testing. You can help yourself with this calculator.
- Test the hypothesis. After performing the A / B test, compare the results with the hypothesis you have identified and critically evaluate whether the test results have confirmed or refuted this hypothesis.
- Analyze the data and draw final conclusions. If the additional links have reduced the bounce rate, then you can confirm that the number of extra footer links has affected the bounce rate on the website. However, if there is no significant change in the conversion rate after A/B testing, then return to point three and set a new hypothesis for the new test.
Your first A / B test
There’s a bunch of tools available. We use Google Optimize for the simple tests. It’s intuitive and easy to set up. We have written a longer post with instructions: How to do A/B Tests With Google Optimize.
- Insert the tracking code into the web page
- Measure the elements on the website you want to test
- Define A/B Testing
Select the objectives for which you want to increase the conversion rate. There can be only one goal. The goal can be to fill out a contact form, sign up for an e-newsletter, download a free guide… What matters most to you at the moment.
- Start and measure the results
The A / B testing tool allows you to see the results of the current statistics , as the measurement will start as soon as the first visitors arrive on the website.
- Analyze the results
Only on rare occasions the test will produce clear winners. Usually some statistical manipulation of the data is needed to prove that the test results are valid. Don’t hesitate to seek external help here – a wrong conclusion can produce substantial business costs.
Experiment and have fun!
A/B testing connects lots of different aspects of digital design. And it takes time to master. But the beauty of it is, that you are always safe – if your hypothesis is disproven, you can always return to the original design and try something else.
There are some basic mistakes you should avoid. We’ve collected the most common ones in a blogpost titled The Most Common A/B Testing Mistakes. Check it out, it’s short and to the point.
We would also recommend taking some more time and going through these two great posts published on the CXL website:
And if you want even more, we would recommend CXL’s course Become great at A/B testing. It’s not the cheapest, but you will not regret it.