Enter your control and variant data. Get statistical significance, power, MDE, and the sample size you need — so you never kill a winner or back a loser too early.
How many impressions do you need per variant to reliably detect an improvement of a given size?
The probability that the difference you're seeing is real and not random noise. At 95% confidence, there's only a 5% chance you're wrong. Shown as a p-value — p < 0.05 means significant at 95%.
The probability of detecting a real effect when one exists. Low power (e.g. 30%) means you're likely to miss real improvements. Standard target is 80%+ before drawing conclusions.
The smallest improvement your test can reliably detect at the chosen power level. If your MDE is 25%, you can't reliably spot improvements smaller than that — you'd need more data.
How many impressions per variant you need before the test can detect your target improvement at 80% power and 95% confidence. Running a test without enough data is guesswork.
This tool validates your tests. Shev Dilay helps you design the creative strategy behind them. Technical SEO, content architecture, and AI search visibility for B2B SaaS brands.
Work with Shev →