Free tool
A/B test significance calculator
Paste in your two variants' visitors and conversions to see the conversion rates, the uplift, and whether the result is statistically significant. It is a two-proportion z-test calculator: it reports the z-score and the two-sided p-value as a confidence level, the same test AB Test WP Pro runs on live results.
-
Enter your numbers above.
Your inputs are saved in the page address.
How to read this
Statistical significance tells you how confident you can be that the difference between A and B is real, and not just random chance. The convention is to look for 95% confidence (a p-value below 0.05) before declaring a winner.
- Conversion rate - conversions ÷ visitors for each variant.
- Uplift - how much better (or worse) B did, relative to A.
- Confidence - the chance the observed difference isn't a fluke. Below your threshold means "keep running - not enough evidence yet."
A worked example
Say variant A had 5,000 visitors and 150 conversions (3.0%), and variant B had 5,000 visitors and 190 conversions (3.8%). That's a 26.7% relative uplift, and the two-proportion z-test puts the two-sided p-value around 0.03 - significant at 95%. With 150 vs 165 conversions instead, the same uplift direction would not be significant: the difference is small enough that chance explains it.
Related reading: how long to run an A/B test and A/B testing statistics made simple.
FAQ
Questions
What test does this calculator use?
A pooled two-proportion z-test, two-sided - the standard frequentist test for comparing two conversion rates. It's the same test AB Test WP Pro runs on your live results.
What does 95% confidence mean?
Roughly: if there were truly no difference between the variants, a result at least this extreme would show up less than 5% of the time by chance. It's strong evidence of a real difference - not a guarantee, and not the probability that B is better.
Why isn't my result significant?
Usually the sample is too small for the size of the effect. Keep the test running to a pre-planned sample size - the sample-size calculator tells you what that is - and avoid stopping the moment the number looks good, which inflates false positives.
Get significance on autopilot
Run real tests in WordPress and let Pro tell you the moment a result is solid.