If this error persists, please let us know at our support page.
Our engineering team has been notified.
Goodbye Traditional Statistics
Statistics for the Internet Age is Here
Watch Video
Data has changed, Statistics should too
Traditional statistics were developed over 100 years ago, for an offline world. These methods are no longer up to the task of helping businesses make decisions with real-time data. If you look at results in real time, as most optimization platforms allow you to do, you'll see winning results where no winner actually exists. In fact, your error rate can skyrocket to over 30% if you don't take precautions.
Without taking precautions, like setting a sample size and committing to a minimum detectable effect in advance, you could make changes to your website or product that at best, have no impact, and at worst, could be hurting your bottom line.
Read the blogWe believe statistics
should work the way you do
Our new Stats Engine supports experimentation and decision-making on your terms--no precautions needed. We partnered with Stanford statisticians to develop a revolutionary new approach to statistics for experience optimization. The approach builds on everything we've learned from helping the world’s leading brands optimize hundreds of billions of experiences.
Download the ebook
Introducing always-valid results
Results are valid whenever you check
With traditional statistics, peeking at your results any time other than your set sample size increases the chance you’ll find a winner when there isn’t one. With Stats Engine, check whenever you want. With a method called sequential testing, you’ll see your statistical significance increase with more time and data—no more waiting.
Experiment with as many variations and add as many metrics as you want
Adding many metrics or variations to an experiment also increases your chance of finding a winning result that isn’t real. Optimizely’s Stats Engine corrects for multiple goals and variations with false discovery rate control. Now you can add as many as you want without sacrificing accuracy.
Learn The Math KDD PaperOptimizely's Stats Engine increases confidence
With Stats Engine
setting a 95% significance
level results in a
95%
chance you'll make the right decision
With traditional statistics
setting a 95% significance
level results in a
<70%
chance you'll make the right decision if you don't avoid pitfalls*
* We simulated millions of A/A tests under both frameworks and calculated error rate by determining the percentage of experiments that declared a winning or losing variation given a continuous monitoring policy. (1000+ visitors)
Interested in how Stats Engine affects your results?
Learn moreStatistics has never been easier
Always-valid Statistics
Make a statistically sound decision whenever you look at your results, as soon as data is available.
Significance on Your Schedule
No need to commit to a sample size or duration in advance. You can also see conclusive results up to twice as fast.
Leave the Stats to Us
No need to change how you experiment. Stats Engine works exactly the way you expect it to, without any extra work.
Always-valid Statistics
Make a statistically sound decision whenever you look at your results, as soon as data is available.
Significance on Your Schedule
No need to commit to a sample size or duration in advance. You can also see conclusive results up to twice as fast.
Leave the Stats to Us
No need to change how you experiment. Stats Engine works exactly the way you expect it to, without any extra work.