A/B testing

Statistical Power Calculator

The probability that your study finds an effect that is really there — the number that decides whether it is worth running.

Also called power analysis calculator, statistical power calculator, G*Power alternative.

Updated August 2026Runs in your browser — nothing is uploadedVerified against R

How to read this result

Power is the probability of detecting an effect that is really there. 80% power means that if the effect is as large as you assumed, four studies in five will reach significance and one will not. That 20% is the type II error rate, β.

Underpowered studies are worse than they look. They miss real effects — but they also exaggerate the ones they find, because with low power only an unusually large sample difference clears the threshold. A significant result from a 30%-powered study is likely to overstate the effect substantially.

There are only three levers: the sample size, the effect you are trying to detect, and α. You cannot raise power by analysing more cleverly. Halving the effect you want to detect roughly quadruples the sample required.

Compute power before the study, never after. "Observed power" calculated from your own p-value is a one-to-one function of that p-value and tells you nothing new — a non-significant result always yields low observed power, by construction.

Left curve: what the statistic does when nothing is going on. Right curve: what it does when the effect is real. α is the sliver of the left curve past the threshold — a false positive. β is the part of the right curve that falls short — a miss. Power is everything to the right of the line under the second curve.

The formula

power = P (T > t_{α / 2, ν} | ncp = δ \sqrt{\frac{n}{2}})

$δ$: the effect size, Cohen's d
$ν$: the degrees of freedom, 2(n − 1) for two samples
$n c p$: the non-centrality parameter — how far the alternative distribution sits from the null

Computed from the non-central t distribution, matching R's power.t.test. Calculators that use the normal approximation report slightly too much power at small n.

Worked example

Sixty-four per group, medium effect

The classic pairing in every methods textbook: a medium effect size (d = 0.5) and 64 observations per group. What power does that give at α = 0.05?

Degrees of freedom for two independent samples.
2 × (64 − 1) = 126
The non-centrality parameter.
0.5 × √(64/2) = 0.5 × 5.657 = 2.828
The two-tailed critical value on 126 df.
t* = 1.979
Power is the share of the non-central t distribution beyond that.
P(T > 1.979 | ncp = 2.828) = 0.8015

power 80.1%β (miss rate) 19.9%n for 80% 64 per groupncp 2.828

This is why “about 64 per group” is the number people quote for a medium effect. For a small effect (d = 0.2) the same 80% power needs 394 per group — nearly six times as many for an effect two and a half times smaller.

Checked against R's power.t.test(n = 64, delta = 0.5, sd = 1).

The calculator above is loaded with these numbers by the Load the worked example button.

Assumptions, and when to use something else

Power calculations depend entirely on an assumption you have to supply: how large the effect is. It is not in the data, and getting it from a pilot study is unreliable, because pilot estimates are noisy and biased upward when they were significant.

The defensible approaches are to use the smallest effect that would be worth detecting — a business or clinical judgement — or an effect size typical of the field from published work.

Everything else — normality, independence, equal variances for the pooled design — is inherited from the test the power refers to.

You want the sample size, not the powerSample size calculatorSolves the same equation for n.
You want to know what your existing sample can detectMinimum detectable effectThe smallest effect this design can find at your target power.
You need an effect size to plug inEffect size calculatorComputes Cohen's d from data or from summary statistics.
The result was not significantStatistical significance calculatorRead the confidence interval; power explains what the test could have seen.

Questions people ask

What is a good level of statistical power?

80% is the conventional minimum and 90% is common where a miss is costly. Below 50% the study is more likely to miss a real effect than to find it, which is difficult to justify spending money on.

What is the difference between power and significance?

Significance (α) is the false-positive rate: how often you claim an effect that is not there. Power (1 − β) is the true-positive rate: how often you find one that is. Lowering α to be safer against false positives costs power, which is the trade-off the two represent.

Should I calculate power after my study?

No. Post-hoc power computed from the observed effect is a deterministic function of the p-value and adds nothing — a non-significant result always produces low observed power. If you want to know what the study could have detected, use the minimum detectable effect instead.

How do I choose an effect size?

Pick the smallest difference that would change what you do — a business threshold, a clinically meaningful change. Falling back on Cohen's conventions (0.2/0.5/0.8) is acceptable when nothing better exists, and using a noisy pilot estimate is not.