Glossary
Minimum detectable effect (MDE)
The number that trades traffic against sensitivity, how it drives the sample size, and a calculator to set it for your page.
Definition: The minimum detectable effect (MDE) is the smallest improvement a test is designed to detect reliably at its sample size. Choose it before the test: a 10% relative lift on a 4% baseline needs far more visitors than a 30% lift. It is the lever that trades traffic against sensitivity, and the reason low-traffic sites should test bold changes.
Relative or absolute
An MDE is usually relative: "a 20% lift" on a 5% baseline means detecting a move to 6%. Absolute MDEs ("one percentage point") are clearer when baselines differ a lot between pages. Say which one you mean; the sample sizes are very different.
How it sets the sample size
n per variant = (1.96 + 0.84)² × [p₁(1 - p₁) + p₂(1 - p₂)] / (p₂ - p₁)²
p₁ is the baseline, p₂ the baseline lifted by the MDE, 1.96 the z-value for 95% two-sided confidence and 0.84 the value for 80% power. The MDE sits in the denominator, squared: halve it and the sample roughly quadruples. That single fact explains most failed tests.
Choosing an MDE
Ask two questions. What is the smallest improvement that would be worth acting on? And what improvement is plausible for this change? The MDE should sit at or below the first and at or below the second. If the resulting sample is more traffic than the page gets in eight weeks, the change is too small to test here: test something bolder, or a higher-traffic page.
Calculate it for your page
MDE and sample-size calculator
Type your own numbers; the result updates as you type.
You need
16,312visitors in total
About 17 days at your traffic
Two-sided 95% confidence, 80% power, lift relative to the baseline; total = per variant × variants. The full calculator adds a confidence setting and shareable results.
A worked example
Baseline 5%, MDE 20% relative (to 6%): 8,156 visitors per variant, 16,312 in total, 17 days at 1,000 visitors a day. Same baseline, MDE 10% (to 5.5%): about 31,200 per variant, 62,500 in total, nine weeks. A site with 300 visitors a day cannot run the second test in a reasonable time and should design a change big enough to move the rate by 20% or more.
Common mistakes
- Leaving the MDE at a tool's default and being surprised by the visitor count.
- Setting the MDE after the test to whatever the observed lift was.
- Confusing relative and absolute lifts when copying numbers between tools.
- Testing a small change on a low-traffic page and stopping early because the sample looked impossible.
In AB Test WP
AB Test WP ships a sample-size planner with the same formula (two-sided 95% confidence, 80% power, lift relative to the baseline, total = per variant times variants), so the visitor count and the finish date are known before a test starts.
Related: Full sample-size calculator · How many visitors an A/B test needs · all glossary terms.
Common questions
What MDE should I choose?
The smaller of what would be worth acting on and what the change could plausibly deliver, checked against the traffic you have. For most pages 10% to 30% relative is the workable range.
Relative or absolute?
Relative is the convention in A/B testing tools and in this calculator. A 20% relative lift on a 5% baseline is a move to 6%, that is one absolute point.
What if my traffic cannot reach the sample?
Raise the MDE by testing a bolder change, choose a more frequent goal, test on a higher-traffic page, or accept a lower confidence level for a cheap, reversible change and say so in the write-up.
Run the numbers on your own test
Free calculators, no email wall: significance, sample size, conversion rate and SRM.