Revenue Impact
Revenue impact translates A/B test results into euros — catching the tests that lift conversion rate but quietly tank AOV or margin.
Revenue Impact
The euro (or dollar) lift an A/B test variant generates per visitor, combining conversion rate and average order value into one number.
Revenue impact is the bottom-line result of an experiment, expressed as revenue per visitor (RPV) or as annualised incremental revenue. Instead of celebrating a conversion-rate win in isolation, you multiply CR by AOV to get the metric your CFO actually reads.
This matters because conversion rate and order value frequently move in opposite directions. A discount banner can lift CR by 6% while dragging AOV down 9% — a net revenue loss disguised as a win. Revenue impact is the guardrail that catches that scenario before you ship the variant store-wide.
Most teams declare an A/B test winner the moment conversion rate hits statistical significance. That is the cheapest mistake in CRO. Conversion rate is a ratio, not money — and ratios can rise while revenue falls.
The classic trap: a free-shipping threshold change lifts checkout completion by 5% but pulls AOV down by 8% because shoppers add fewer items. CR is up, your dashboard glows green, and quarterly revenue is down four figures a week. Revenue impact analysis is the layer of experiment analysis that prevents this.
Revenue Impact = (RPV_variant - RPV_control) * Annual Visitors
RPV_variant
Revenue per visitor (variant)
Total variant revenue divided by variant visitors during the test.
RPV_control
Revenue per visitor (control)
Total control revenue divided by control visitors during the test.
Annual Visitors
Annualised traffic
Visitors expected to see the winning experience over a full year if shipped.
A Shopify apparel store tests a new product-page layout. Over 21 days, control sees 42,000 visitors at 3.2% CR and €78 AOV. Variant sees 42,000 visitors at 3.6% CR but AOV drops to €71. The store gets 1.1M visitors a year on that template.
RPV control: 0.032 × €78 = €2.50
RPV variant: 0.036 × €71 = €2.56
Annual visitors: 1,100,000
→ (€2.56 − €2.50) × 1,100,000 = €66,000 annual revenue lift
CR rose 12.5% but AOV dropped 9% — net revenue is only +2.4%, not the +12.5% the CR-only readout suggested. Still a ship decision, but worth a fraction of what the headline implied.
The same arithmetic flips negative more often than teams expect. Aggressive discount banners, urgency timers on already-discounted SKUs, and simplified cart flows that strip cross-sell modules tend to lift CR while quietly suppressing AOV. Always compute RPV deltas, not CR deltas alone.
How CR and AOV shifts combine into net revenue impact
| Scenario | CR change | AOV change | Net revenue impact |
|---|---|---|---|
| Free-shipping threshold lowered | +5% | −9% | −4.4% |
| PDP redesign with clearer sizing | +3% | +2% | +5.1% |
| Aggressive sitewide discount banner | +8% | −12% | −5.0% |
| Express checkout (Shop Pay) prominence | +4% | 0% | +4.0% |
| Bundle recommendation in cart | −1% | +11% | +9.9% |
Two operational rules follow. First, set revenue per visitor as your primary experiment metric and CR as a supporting one inside your experiment analysis — not the other way around. Second, segment revenue impact by device and traffic source: a variant that wins on desktop paid search can lose on mobile organic, and the blended number hides it.
Revenue impact FAQ
Conversion rate ignores order value. A variant can win on CR while losing on revenue if AOV drops more than CR rises. Revenue per visitor combines both into the number your P&L actually tracks.
RPV is the per-visitor unit (revenue ÷ visitors). Revenue impact is the projected total — the RPV delta multiplied by annualised traffic. RPV is how you compare variants; revenue impact is how you justify shipping to a CFO.
It sits at the top of the metric hierarchy. Your experiment analysis should read revenue impact first, then CR and AOV as diagnostics that explain why revenue moved, then secondary metrics like add-to-cart and bounce.
Yes. RPV has higher variance than CR because order value distributions are skewed by large orders. Test for significance on RPV directly, not just CR — otherwise you ship variants that look revenue-positive but aren't statistically distinguishable from noise.
Trim or cap the top 1% of orders before computing RPV. A single €4,000 order during a two-week test on a beauty store can swing the entire result. Most experimentation platforms offer winsorisation as a built-in option.
Gross revenue for most tests, contribution margin when the variant changes discount depth, shipping subsidy, or product mix. A free-shipping test that lifts gross revenue 3% but absorbs €4 shipping per order is often margin-negative.
Long enough to cover at least two full purchase cycles for your category — typically 2-4 weeks for apparel and beauty, longer for considered electronics. RPV stabilises slower than CR because high-value orders are rare events.
Routinely. The most common pattern is a friction-removing change that increases checkout completion among low-intent browsers, dragging AOV down because those incremental buyers purchase fewer or cheaper items.
Take the RPV delta from the test window and multiply by realistic annualised traffic on the affected template — not all sitewide traffic. Discount the projection 20-30% to account for novelty effects and seasonal drift.
Always. Mobile and desktop visitors convert and spend differently, and paid traffic often behaves differently than organic. A variant with +2% blended revenue impact can be +8% on desktop and −5% on mobile — which means you ship it as a desktop-only treatment, not sitewide.
Test ideas before you ship them
Run unlimited A/B tests, attach hypotheses to outcomes, and build a searchable archive of what works — and what doesn't.