Introduction
A/B testing is considered the secret sauce that can help you with conversion rate optimisation, get more out of your existing traffic, and grow your audience.
You need to have a clear strategy to ensure your A/B testing efforts give you the most bang for your buck, rather than running random A/B tests on your website with no clear direction that will ultimately generate random, and most likely, unsatisfactory results. Keeping that in mind, here are a few questions that you should ask yourself about A/B testing.
Quality or Quantity Or Both
It is key to prioritise A/B testing efforts in line with intended data outcomes. However it’s important to understand that not every A/B test you run is going to lead to actionable results.
Whether you’re just starting off or have an established business, if you are seeking a reasonably quick improvement to your sales you should be focused on running high-quality A/B tests, rather than large quantities of tests at random. Running tests with intention is pivotal in yielding results that will generate an uplift in sales. One of the most common mistakes that businesses make is increasing the number of tests as evidence for annual company growth. For clarity, running more tests is great but increasing the number of tests has to be done once the right foundations are in place, the saying “walk before you run” sums it up nicely!
You should focus on analysing and reflecting on your testing methods, and tracking results. Major indicators include win rate, Return on Investment, and the implementation rate – all of which are indicators towards a better understanding of which A/B tests you should focus on.
Inconclusive Tests
Individual Segments
Breaking down your tests into individual segments can reveal overlooked elements of data. It is also vital to understand test performances across multiple segments of the testing phase, such as traffic sources and devices, along with any other segments that make sense to you against verticals.
The key to conclusive tests is to refrain from carrying out dumb tests. In other words, if the changes are too minute, the results of your A/B tests will almost always come back as inconclusive.
Be Bold with Your A/B Tests
One of the best ways to be bold with A/B tests is to not stress over the first test, but rather, use it to attune your hypothesis. Is your hypothesis wrong or incomplete? Do you need to make changes to your target audience, or evaluate processes?
Bold actions are necessary for reaping greater results in A/B testing. In order to decide what bold action to take, you must first identify whether larger tests, personalisation, or smaller, incremental test ideas are the appropriate approach.
Other ways of being bold with your A/B tests includes setting goals based on consistency, and being more open to optimization.
Statistical Significance and Low-Volume Projects
The best strategy for small businesses and start-ups with a small audience to use when it comes to A/B testing is to wait until traffic has increased. Once you have a good number to work with, you can then use A/B testing to boost your conversion rate.
While there are some who suggest that Statistical Significance needs to be reduced in order to push for outcomes of low-volume testing, there are still others who advocate for taking a more balanced approach, especially if you’re working with limited data.
For instance, if you have an ecommerce store that is just able to generate 5 to 10 sales from 2,000 monthly visitors, A/B testing should not be carried out until you’ve gathered a sufficient amount of data, which usually takes a few months. Then again, there are others who suggest that A/B testing is not useful in the case of small audiences.
It is also important here to note that A/B testing is not only about increasing the conversion rate of your website, but it is also about gaining a deeper understanding of your audience.
