A/B testing

Sample Size Calculator

How many observations you need — decided before you collect any, which is the only time it can be decided.

Updated August 2026Runs in your browser — nothing is uploadedVerified against R

Margin of errorAs a decimal: 0.03 for ±3 percentage points.

Expected proportion (optional)Leave empty for the conservative 0.5, which needs the largest sample.

Population size (optional)Only matters if you are sampling a large fraction of it.

Confidence level (%)

The calculation runs in your browser, so this box needs JavaScript. The formula, the worked example and the interpretation below do not.

How to read this result

Sample size is decided before data collection, from three inputs: how precise you need to be, how variable the data are, and how confident you want to be. Changing any of them afterwards is not a calculation, it is a rationalisation.

Precision costs quadratically. Halving the margin of error takes four times the sample; halving the effect you want to detect takes four times the sample. That relationship is why studies get expensive so fast and why sample sizes cluster around a few conventional numbers.

Round up, always. Rounding down leaves the margin slightly wider or the power slightly lower than you asked for.

This is the number you need to analyse. Divide by your expected response or completion rate to get the number to invite: 1,067 completed surveys at a 30% response rate means about 3,600 invitations.

The formula

For a survey proportion

n = \frac{z^{2} p (1 - p)}{e^{2}}

$e$: the margin of error you want
$p$: the expected proportion; 0.5 is the conservative choice

With a finite population correction n′ = Nn/(n + N − 1) when the population is small.

For comparing two rates

power = Φ (\frac{\sqrt{n} | p_{1} - p_{2} | - z_{α / 2} \sqrt{2 \bar{p} \bar{q}}}{\sqrt{p_{1} q_{1} + p_{2} q_{2}}})

Solved numerically for n, matching R's power.prop.test.

Worked example

How many visitors to detect a 10% to 13% lift

Your conversion rate is 10%. You want to be able to detect an improvement to 13% — a three-point, 30% relative lift — with 80% power at the 5% level. How much traffic does each variant need?

The two rates and their average.
p₁ = 0.10, p₂ = 0.13, p̄ = 0.115
The z values for α and for power.
z_{α/2} = 1.960, z_power = 0.842
Solve the power equation for n.
n = 1,773.98 per group
Round up, and double for the total.
1,774 per variant, 3,548 in total

per group 1,774total 3,548power 80%at 90% power 2,374 per group

Detecting a smaller lift — 10% to 11% — would take about 14,700 per group. That is the quadratic cost of precision, and it is why deciding the detectable effect in advance is the most consequential choice in an experiment.

Checked against R's power.prop.test(p1 = 0.10, p2 = 0.13, power = 0.8).

The calculator above is loaded with these numbers by the Load the worked example button.

Assumptions, and when to use something else

A random sample from the population you want to describe. No sample size compensates for a biased frame.
An honest estimate of variability. For a mean, the answer is entirely driven by the assumed standard deviation; get it from a pilot or from published work, and treat the result as approximate.
One analysis at the end. Sequential monitoring needs a different design.
Equal group sizes, for the comparison mode. Unequal allocation needs more total sample for the same power.

You want the power of a sample you already havePower calculatorThe same equation solved the other way.
You want to know what your existing sample can detectMinimum detectable effectThe practical version of the question when the sample size is fixed.
You want the margin of error for a given nMargin of error calculatorThe precision that comes with a sample you have.
You are ready to read the resultA/B test calculatorFor when the experiment has finished.

Questions people ask

What sample size do I need for a survey?

For ±3 percentage points at 95% confidence, about 1,067 completed responses — regardless of whether the population is a town or a country, provided it is much larger than the sample. For ±5 points, 385. For ±1 point, 9,604.

Why does the population size barely matter?

Because precision depends on how many people you asked, not on how many you could have asked. The finite population correction only bites once your sample is more than about 5% of the whole population.

How do I choose the effect size to detect?

Pick the smallest improvement that would change your decision. If a 1% lift would not justify shipping the change, do not size the study to detect it — size it for the smallest lift you would act on.

What if I cannot get that many observations?

Then compute the minimum detectable effect for the sample you can get. If it is far larger than any plausible effect, the study will not answer the question, and knowing that before running it is worth a great deal.