Loading page
About this tool
Calculate the p-value and confidence level from your A/B test conversion data.
The A/B Test Significance Calculator runs a two-sided pooled two-proportion z-test on an A/B test: enter visitors and conversions for each variant and it returns the z score, the exact p-value, each variant's conversion rate and the relative uplift. It pools the two samples to estimate the standard error, so the verdict is the standard frequentist one used by most experimentation platforms. You choose 90%, 95% or 99% confidence, and the result is called significant when p is below the matching alpha of 0.10, 0.05 or 0.01.
Open A/B Test Significance Calculator on AltFTool — it loads instantly in your browser.
Enter your test counts in the Variant A visitors, Variant A conversions, Variant B visitors and Variant B conversions fields (defaults 1000/100 vs 1000/125).
Pick 90%, 95% or 99% in the Confidence target (%) select - this sets the alpha the two-sided p-value is compared against.
Read the live verdict - Statistically significant or Not significant at target - with the p-value to six decimals, z score, both conversion rates and relative uplift; Copy or Download saves a-b-test-significance-calculator.txt.
You get the z score and the p-value to six decimal places, so you can see how close a borderline result really is.
Both conversion rates and the relative uplift are reported, which stops a 0.2-point move on a tiny base from looking like a win.
Switching between 90%, 95% and 99% changes the alpha the p-value is compared against, so the threshold behind the call is never hidden.
Below your chosen alpha: 0.05 at 95% confidence, 0.10 at 90%, and 0.01 at 99%. A p-value of 0.03 clears the bar at 95% but not at 99%. The p-value is the probability of seeing a difference at least this large if the two variants actually converted at the same rate — it is not the probability that B is better.
You can look, but do not decide. This is a single planned analysis: repeatedly checking and stopping the moment p dips under 0.05 inflates the false-positive rate well beyond the nominal 5%. Fix the sample size in advance, or use a sequential testing method designed for peeking.
Almost always sample size. The standard error scales with the square root of traffic, so at a 10% baseline a few hundred visitors per arm can only detect very large swings. A 10% to 12.5% move needs roughly a thousand visitors per variant to reach 95% confidence; smaller true effects need several times that.
No. The pooled z-test assumes each visitor is a yes/no conversion, so it applies to binary outcomes like signups, purchases or clicks. Continuous metrics such as revenue per visitor or session duration are skewed and need a t-test, a bootstrap, or a test on a transformed metric instead.
Add the A/B Test Significance Calculator widget to your blog or website — free, responsive, no signup. Just keep the “Widget by AltFTool” credit link visible.
<iframe src="https://www.altftool.com/embed/widget/a-b-test-significance-calculator"
title="A/B Test Significance Calculator — free AltFTool widget"
width="100%" height="640" style="border:0;border-radius:12px;overflow:hidden"
loading="lazy" referrerpolicy="no-referrer-when-downgrade" allow="clipboard-write"></iframe>
<p style="font-size:12px;margin:4px 0 0">Widget by <a href="https://www.altftool.com/tools/all/a-b-test-significance-calculator?utm_source=embed&utm_medium=widget" rel="nofollow">AltFTool — free online tools</a></p>