A/B testing
Effect Size Calculator
How big the difference is — the question a p-value cannot answer.
Have a spreadsheet? Drop a CSV here
Or . CSV, TSV or plain text. You pick which column is which after it loads — the names in your file do not have to match ours. The file is read in your browser and never uploaded.
How to read this result
Cohen's d is the difference in standard deviations. A d of 0.5 means the two means are half a standard deviation apart, whatever the original units — which is what makes effect sizes comparable across studies and scales.
Unlike a p-value, it does not grow with the sample. That is the whole point: a huge study finds a d of 0.02 highly significant, and 0.02 is still nothing.
The conventions are conventions. 0.2 small, 0.5 medium, 0.8 large — Cohen offered them reluctantly, for fields with no established benchmarks, and warned against applying them blindly. Where your field has typical effect sizes, compare against those.
The interval on d is usually wide. Effect sizes are estimated far less precisely than people assume: a study that just reaches significance typically has a d interval running from "negligible" to "large". That width is not a flaw in the calculation.
Hedges' g is d with the small-sample bias removed. Below about 20 per group, prefer it — the correction shrinks d by a few per cent.
The formula
- the pooled standard deviation
- the difference expressed in standard deviations
Small-sample correction
Hedges' g. The correction is about 4% at n = 10 per group and negligible past 50.
Worked example
Two groups of five
Five observations in each of two groups. The means differ by about 2.1 units and the pooled standard deviation is 0.775 — so the difference is large relative to the spread.
Group 1: 12.9, 13.4, 11.9, 14.2, 12.8 · Group 2: 15.1, 15.2, 14.1, 16.0, 15.5
- Both means.13.04 and 15.18
- Both variances, then pool them.s₁² = 0.713, s₂² = 0.487 → sₚ² = 0.600 → sₚ = 0.7746
- Divide the difference by the pooled SD.d = (13.04 − 15.18) / 0.7746 = −2.763
- Apply Hedges' correction for the small sample.g = −2.763 × (1 − 3/31) = −2.496
- Translate into something readable: the share of pairwise comparisons group 2 wins.about 97%
A d of 2.76 is enormous — the two distributions barely overlap. Effects this size are rare outside contrived examples and usually indicate the groups differ in something more fundamental than the treatment.
Checked against the pooled-SD definition, computed in R.
The calculator above is loaded with these numbers by the Load the worked example button.
Assumptions, and when to use something else
Cohen's d assumes the two groups have comparable variances, since it pools them. When they do not, Glass's delta — which divides by the control group's SD alone — is the better choice, and is reported alongside.
The confidence interval uses the standard normal approximation to the sampling distribution of d. It is adequate for moderate samples and slightly narrow for very small ones; exact intervals from the non-central t are marginally wider.
Effect sizes for paired designs are a genuine trap: d computed from the paired differences (dz) is larger than d computed from the two groups' pooled SD, and papers rarely say which they used.
- You want to know whether the difference is realT-testThe significance test that pairs with this effect size.
- You are planning a studyPower calculatorEffect size is the input that decides how many observations you need.
- The outcome is binaryOdds ratioFor yes/no outcomes the odds ratio and relative risk are the effect sizes.
- You are comparing more than two groupsOne-way ANOVAη² and ω² are the corresponding effect sizes, reported with every ANOVA here.
Questions people ask
What is a large effect size?
By Cohen's conventions, d = 0.8. But they are rules of thumb for fields with no benchmarks: in medicine a d of 0.2 can be enormously valuable, and in a physics calibration 0.8 would suggest something is broken. Compare against typical effects in your own area.
Cohen's d or Hedges' g?
Hedges' g below about 20 observations per group, where d is biased upward. Above that they are practically identical. Report whichever you use, by name.
Can I get an effect size from a published t-test?
Yes — the third mode does exactly that. d = t × √(1/n₁ + 1/n₂), which recovers the effect size from a t statistic and the two group sizes.
Why is my confidence interval for d so wide?
Because effect sizes are estimated imprecisely. With 20 per group, a d of 0.5 has a 95% interval running roughly from −0.13 to 1.12. That is not a computational problem; it is what forty observations can tell you.