A/B testing tools are a powerful method of making informed decisions and making improvements to a specific set of key performance indicators (KPIs).
A/B testing was commonly used for direct mail (the paper variety) prior to the internet. In 1923 Claude C. Hopkins wrote a book called Scientific Advertising which is often cited as the most original document that not only describes “split testing” (A/B testing) but also coupon tracking.
By tracking metrics based on the durability of their wares, manufacturers and quality engineers use forms of multivariate testing to improve the quality of their merchandise.
In the information age, every brand with a website can gain an intimate understanding of their audience and user behavior through direct usability testing and A/B testing methods. With these methods, they can begin experimenting with thousands of users and pivot their experiments to audience reactions instantaneously.
As the old adage goes, “garbage in, garbage out.” No matter how complex, powerful and algorithmically-sound your testing tool is, if you do not have a strong reason to believe your actual ideas will work, then don’t use them.
In fact, it would behoove any brand to look at the data that is available to them and construct a long-term testing plan based on research before they even look at a testing tool interface.
The data available in web analytics is expansive, but it can be impersonal when you want to determine the deeper psychology of your users outside of finding a conversion chokepoint. This is when usability testing becomes valuable. By utilizing a lab, survey tool, or even your own employees, there are several methods that can be used to both generate good testing ideas and remove bad testing ideas (i.e. ideas unlikely to succeed).
Card Sorting – Utilizes a group of experts or users to brainstorm examples of ideal streams of user behavior. This can utilize software or simple index cards.
Moderated In-Person Usability Tests – Record the user behavior and reactions of a person as they attempt to move through a specific flow. There are also remote tests that leverage screen sharing software as well as unmoderated platforms that can help gather information quickly.
UX Surveys –Making a survey available to your website users can help build the narrative around why certain behaviors are being observed in analytics.
No matter which method you choose to learn more about your users, remember: if you can link together the data shown by web analytics with the results of the UX test, you’ll find the most solid results.
Once you have determined what the most insightful tests will be, you must plan them out and build accordingly. Make sure that each test is measuring the correct KPI and to watch other metrics that may be affected. Keep in mind that some of your tests will require technical resources when prioritizing your testing schedule.
When building your tests, it is critical to integrate the test experience with your web analytics platform. This will enable you to match up the data found natively on your testing platform with your main web analytics data warehouse (it is unlikely that they will match up directly). This is very valuable to companies that have already invested in advanced custom web analytics tracking.
In summary: