A/B Test Significance Calculator
Compare visitors and conversions for A and B to get the p-value, lift and whether the A/B test result is significant.
Use A/B Test Significance Calculator
Assumes randomly assigned visitors, one outcome each and one planned comparison, with no adjustment for peeking. Free, no sign-up. Runs in your browser; nothing you enter is uploaded.
How to use it
Check if the gap between two conversion rates in an A/B test is real or could be chance.
- Enter unique visitors and conversions for A, your control.
- Enter the same counts for B, over the same dates.
- Pick the confidence level you chose before the test.
- Decide at your planned sample size or end date, not at the first significant result.
Formula
Pooled rate p̂ = total conversions / total visitors. z = (rate B − rate A) / √(p̂(1 − p̂)(1/visitors A + 1/visitors B)). p = 2 × Φ(−|z|)
Lift = (rate B − rate A) / rate A
Example
India · ₹1,499 kurta product-page test
A D2C brand compares two product-page layouts by orders.
- Visitors in A
- 12,000
- Conversions in A
- 360
- Visitors in B
- 12,000
- Conversions in B
- 432
- Confidence level
- 95%
Questions
What does statistically significant mean here?
If the versions had equal underlying conversion rates, a gap at least this extreme would be unlikely under the test model. At 95% confidence, p must be at most 0.05. This is not a 95% probability that B will win, or a guarantee of future revenue.
Why does the calculator need more data?
The pooled z-test uses a normal approximation. We require at least 5 expected conversions and 5 expected non-conversions in each version at the pooled rate. The visitor minimum only checks this condition; it is not a sample-size or power recommendation.
Which confidence level should I choose?
95% is the default. 99% requires stronger evidence; 90% accepts a higher false-positive rate for a single planned test. Choose before seeing results, rather than switching levels to claim a win.
Can I stop as soon as the result is significant?
Repeated checks followed by early stopping increase false-positive risk. Set the sample size or end date before testing. This calculator does not adjust for repeated looks or multiple comparisons.
How is lift different from percentage-point difference?
A change from 3% to 3.6% is a 0.6 percentage-point increase and a 20% relative lift. When A has no conversions, relative lift is undefined, but the point difference still shows.
Can I use impressions or repeat purchases as visitors and conversions?
No. Use independent people, randomly assigned to just one version, and at most one conversion per person. Use the same goal and date range for both versions. Repeated impressions or purchases break this model.
Sources
- NIST: comparison of two proportions (opens in a new tab)
Pooled two-sided z-test. Checked 26 Oct 2026.
- NIST: normal-approximation check (opens in a new tab)
Needs 5+ expected conversions and non-conversions per version. Checked 26 Oct 2026.
Related tools
Turn a product URL into a full ad campaign.
1,000 free credits, no card.