What is Split Testing?

Optimizely: Split Testing Simplified

Split testing (also referred to as A/B testing or multivariate testing) is a method of conducting controlled, randomized experiments with the goal of improving a website metric, such as clicks, form completions, or purchases. Incoming traffic to the website is distributed between the original (control) and the different variations without any of the visitors knowing that they are part of an experiment. The tester waits for a statistically significant difference in behavior to emerge. The results from each variation are compared to determine which version showed the greatest improvement.

This marketing methodology is frequently used to test changes to signup forms, registration pages, calls to action, or any other parts of a website where a measurable goal can be improved. For example, testing changes to an online checkout flow would help to determine what factors increase conversions from one page to the next and will lead to increased orders for the website owner.

Seemingly subjective choices about web design can be made objective using split testing, since the data collected from experiments will either support or undermine a hypothesis on which design will work best. Demonstrating ROI (return on investment) for a testing platform can be measured easily because tests are created with a quantifiable goal in mind.

Split testing tools allow for variations to be targeted at specific groups of visitors, delivering a more tailored and personalized experience. The web experience of these visitors is improved through testing, as indicated by the increased likelihood that they will complete a certain action on the site.

Within webpages, nearly every element can be changed for a split test. Marketers and web developers may try testing:

• Visual elements: pictures, videos, and colors
• Text: headlines, calls to action, and descriptions
• Layout: arrangement and size of buttons, menus, and forms
• Visitor flow: how a website user gets from point A to B

Some split testing best practices include:

• Elimination: fewer page elements create less distractions from the conversion goal
• Focus on the call to action: text resonates differently depending on the audience
• Aim for the global maximum: test with the overarching goal of the website in mind, not the goals of individual pages
• Provide symmetric and consistent experiences: make testing changes consistent throughout the visitor flow to improve conversions at every step of the process

Habitual testing for a website owner or business helps to build a culture of data-informed decision-making that takes into account audience preferences. Each click on a website is a data point from a potential customer. Conflicting opinions can be put to the test with split testing methodology, and the visitors to the website will inform the final decision on the "best" design.

Split testing is equivalent to performing a controlled experiment, a methodology that can be applied to more than just web pages. The concept of split testing originated with direct mail and print advertising campaigns, which were tracked with a different phone number for each version. Currently, you can split test banner and text ads, television commercials, email subject lines, and web products.

Optimizely is a tool that makes creating, running, and reporting split tests easy. Test it out on your website today.