Glossary
Uplift (lift) in A/B testing
The improvement number, in both of its forms, with the interval that keeps it honest and the calculation that turns it into money.
Definition: Uplift (or lift) is how much better the variant performed than the control, usually as a relative percentage: a conversion rate moving from 3.0% to 3.6% is a 20% relative uplift, or 0.6 percentage points absolute. Because it is measured on a sample, an uplift comes with a confidence interval, and the interval is the honest number.
Relative and absolute
relative uplift = (variant rate - control rate) ÷ control rate × 100
absolute uplift = variant rate - control rate (in percentage points)
Relative uplift is the convention because it is comparable across pages with different baselines. Absolute uplift is what you multiply by traffic to get extra conversions. A "20% uplift" on a 0.5% checkout is 0.1 points; on a 30% add-to-cart it is 6 points.
What an uplift is worth
Extra conversions per month = monthly visitors × absolute uplift. Multiply by the value of a conversion for the revenue. The conversion rate calculator has an uplift field that does this for you.
The interval around it
The observed uplift is a point estimate from a sample. The confidence interval gives the range the true uplift is likely to be in, and it is often wide: a test that reports +25% may have an interval from -1% to +51%. Decide on the interval, not the headline number.
Why declared winners overstate it
If you declare a winner the moment a test crosses the significance line, you are selecting for tests that were lucky as well as good, so the measured uplift is biased upwards. This is the winner's curse, a form of regression to the mean. Expect the real-world gain to be smaller than the test said, and re-measure after the change ships.
A worked example
A page with 10,000 visitors a month converts at 3.0%, 300 orders. A variant converts at 3.6%: relative uplift 20%, absolute 0.6 points, 60 extra orders a month, $3,000 at a $50 average order. If the interval runs from +5% to +35%, the extra orders could be anywhere from 15 to 105; the midpoint is a reasonable planning number, the ends are the risk.
Common mistakes
- Quoting relative uplift where absolute is needed, or the other way round.
- Reporting +25% without the interval.
- Adding the uplifts of several tests together and expecting the sum.
- Forgetting that the winning variant's measured uplift is biased upwards.
In AB Test WP
AB Test WP Pro shows the relative uplift with its confidence interval for each variant; Free shows the rates and the leader so the uplift can be read directly.
Related: Conversion rate and uplift calculator · Why an A/B test winner did not hold · all glossary terms.
Common questions
Is a 5% uplift good?
On a high-traffic page with a large order value, 5% can be a great deal of money; on a small page it may not cover the cost of the test. Convert it to conversions and revenue per month before judging.
Why is the real-world gain smaller than the test said?
Winner's curse and novelty. Tests that stop the moment they cross the line select for lucky results, and new designs often perform better in their first weeks. Re-measure after shipping.
How do I convert uplift into revenue?
Extra conversions = monthly visitors times the absolute uplift; revenue = extra conversions times the average value of a conversion. The conversion rate calculator does both.
Run the numbers on your own test
Free calculators, no email wall: significance, sample size, conversion rate and SRM.