A/B testing is a hot topic in marketing now that data is such an integral part of our marketing efforts.
This method of testing is used to optimize and streamline the user experience on an email, landing page, or web page.
You may understand the concept of A/B testing, you may have even run a few A/B tests on your own. But do you know every one of these surprising A/B testing tips?
Work From the Bottom of the Funnel to the Top
One of the first things you define in A/B testing is your goals. The final goal, of course, is the conversion. This means the purchase typically. Optimize the very end of your funnel first.
Why? These optimizations make the biggest impact. They directly affect your goals and will ensure that your leads aren’t finding friction at the very end of your funnel.
You Need to Know What You’re Looking For
You must have goals. That means figuring out what you want a user to do when they open an email, read a post, or land on your website.
Before you start trying to run A/B tests willy-nilly, you need to know what goals you’re interested in optimizing and which elements of your test page affect those elements.
Start With High Impact Pages
It might be tempting to attempt to optimize for every page, but you need to focus on the highest impact pages, especially when you first get started.
What does that mean? It means:
- Pages near the bottom of the funnel
- Contact pages
- Landing pages for email campaigns
- Your homepage
You Need to Use the Scientific Method
To have a truly successful A/B test, you need to run it like the actual scientific experiment. You can’t just wing it and try to randomly test elements.
You can work off the following list of steps:
- Find problem areas in your funnel — these are areas you want to optimize.
- Find issues using user behavior — Figure out what is stopping users from converting.
- Create a hypothesis— this should be a short sentence that describes a change you think will increase conversions. For example, “Changing the CTA button to red will increase conversions.”
- Test your hypothesis – design a new page with a red button and split your traffic so half goes to the control and half to the test page. Halt your test when it’s clear you have a winner. We’ll go into that more later.
- Analyze the results and draw conclusions — from that A/B test, what did you learn. Did the change you make increase conversions in a statistically significant way? If the test is inconclusive, you might have to restart the test, starting at your hypothesis.
Only Change One Variable At a Time
While it might be tempting to change a bunch of things at once, then you don’t know what your users prefer. If you change the size of the button and the color of the button and you see increased conversion, which of those variables caused it.
There’s no way to tell in a simple, single variable A/B test. Just make one change at a time. You’ll learn what your users prefer, which will optimize both the page you’re working on and other pages you may work on in the future.
Start Fresh For Each Test
If you do a test on one variable, then add another, be sure to start a new test. You don’t want to just add a new variable and call it a day.
It will make it almost impossible for you to determine when to stop the test. On top of that, you won’t be able to determine which of the variables worked and which didn’t.
Keep it simple and test one thing at a time.
You Need to Test For Long Enough
How long is long enough for an A/B test?
There are four factors to running a “long enough” A/B test.
- You reach a sufficient sample size. This will have to be calculated before the test occurs. There’s a tool for this.
- The test has run for multiple sales cycles (think 2-4 weeks)
- You reach statistical significance. That’s a formula that determines whether a result is from chance or from fact. The higher the percentage, the more likely the result is the real deal. You can download an extension that will allow you to calculate this on Google Analytics, or you can work with the formula in Excel and find it yourself.
Don’t Cloak Your Test
Cloaking is the practice of showing one set of content to visitors and another to Googlebot.
This can have devastating effects on your website; it can even get your site demoted or removed from Google search results.
To avoid this, don’t serve the test based on user-agent.
Use 302s Instead of 301s
If you’re redirecting traffic from the original URL to the test URL, use a 302 redirect. It’s temporary.
Don’t make the mistake of using a 301 redirect because that’s the permanent one. Your test is temporary, so you don’t want to permanently change where the original URL goes
Let us know what you think:
- Did any of this information surprise you?
- What did you learn from this post?
- Is there anything you think we should add?
