Hypothesis tests

ANOVA Calculator

Q: My groups have different sample sizes. Is that a problem?

Not in itself — ANOVA handles unequal group sizes. It becomes a problem when the variances also differ, because the classic F is then noticeably wrong in a direction that depends on whether the larger group has the larger variance. Use the Welch version, which is reported for every calculation above.

Test whether three or more group means differ, in one test, without inflating the false-positive rate the way several t-tests would.

Updated August 2026Runs in your browser — nothing is uploadedVerified against R

How to read this result

F is a ratio of two variances. The numerator measures how far apart the group means are; the denominator measures how much the observations scatter within each group. If the groups really have the same mean, both estimate the same thing and F sits near 1. A large F means the means are further apart than the within-group scatter can explain.

**The p-value answers one question only: does any pair of means differ?** It does not say which pair, and it does not say by how much. A significant F is a licence to look further, not a finding in itself — follow it with Tukey's HSD, which tests every pair while holding the overall error rate at α.

η² (eta squared) is the effect size. It is the share of the total variation explained by group membership — 0.06 is a modest effect, 0.14 upward is a large one by the usual conventions. ω² (omega squared) is the same idea with the small-sample bias removed, and is the more honest of the two on small designs.

Why not just run t-tests? Three groups make three pairs. Each test at α = 0.05 carries a 5% false-positive risk, so the chance of at least one false positive is about 14%, not 5%. With five groups and ten pairs it is 40%. ANOVA asks the question once, at the level you chose.

The formula

F = \frac{M S_{b e t w e e n}}{M S_{w i t h i n}} = \frac{S S_{b e t w e e n} / (k - 1)}{S S_{w i t h i n} / (N - k)}

$S S_{b e t w e e n}$: \sum n_i(\bar{x}_i - \bar{X})^2 — how far the group means sit from the grand mean
$S S_{w i t h i n}$: \sum (n_i - 1)s_i^2 — the scatter inside the groups
$k$: the number of groups
$N$: the total number of observations

F is compared against the F distribution on k − 1 and N − k degrees of freedom. With exactly two groups, F equals t² and the p-value is identical to a pooled t-test's.

Effect size

η^{2} = \frac{S S_{b e t w e e n}}{S S_{t o t a l}} ω^{2} = \frac{S S_{b e t w e e n} - (k - 1) M S_{w i t h i n}}{S S_{t o t a l} + M S_{w i t h i n}}

η² is the share of variance explained; ω² corrects its upward bias in small samples.

Worked example

Three fertiliser treatments on five plots each

Fifteen plots are split between a control and two fertilisers, five plots each, and the yield of every plot is recorded. The three group means are visibly different — but so are the plots within each group.

Control: 23, 25, 21, 24, 22 · Fertiliser A: 28, 30, 27, 29, 31 · Fertiliser B: 25, 24, 26, 23, 27

Take the group means and the grand mean.
23, 29, 25 → grand mean = 385 / 15 = 25.667
Between-group sum of squares: how far each group mean sits from the grand mean, weighted by group size.
SSB = 5[(23−25.667)² + (29−25.667)² + (25−25.667)²] = 5 × 18.667 = 93.333
Within-group sum of squares: the scatter inside each group.
SSW = 10 + 10 + 10 = 30
Divide each by its degrees of freedom: k − 1 = 2 and N − k = 12.
MSB = 93.333 / 2 = 46.667 MSW = 30 / 12 = 2.5
F is their ratio.
F = 46.667 / 2.5 = 18.667
Look it up on the F distribution with 2 and 12 degrees of freedom.
p = 0.000207
Effect size: the share of total variation explained by treatment.
η² = 93.333 / 123.333 = 0.757

F 18.67df 2, 12p 0.00021η² 0.757

F says something differs. Running Tukey's HSD on the same data shows it is Fertiliser A that stands apart: A differs from both the control and Fertiliser B, and the control and Fertiliser B do not differ from each other.

Checked against R's aov(yield ~ group).

The calculator above is loaded with these numbers by the Load the worked example button.

Assumptions, and when to use something else

ANOVA makes the same demands as the t-test, extended to k groups.

Independent observations, within and between groups. Repeated measures on the same subjects need repeated-measures ANOVA, which is not this test.
Roughly normal within each group — or big enough groups that it stops mattering. Check with Shapiro-Wilk or, better, by looking at the dot plot above.
Comparable variances, for the classic F. Check with Levene's test. When they differ, use the Welch version, which this calculator reports alongside the classic one.
Enough data per group. With two or three observations per group the test has almost no power; a non-significant F from such a design says nothing at all.

Only two groupsT-testWith two groups ANOVA is exactly a pooled t-test — F = t² and the p-values match — but the t-test also gives you the direction and a confidence interval.
F is significant and you need to know which pairs differTukey's HSDTests all pairs while holding the family-wise error rate at α. This is the standard follow-up.
Data are skewed, ordinal, or have outliersKruskal-Wallis testThe rank-based equivalent of one-way ANOVA. Assumes no normality.
The variances are clearly unequalLevene's testConfirms the problem — then use the Welch option above, which handles it.

Questions people ask

What does a significant ANOVA result actually tell me?

That at least one group mean differs from at least one other. It does not say which, how many, or by how much. The next step is a post-hoc test — Tukey's HSD is the standard choice — which compares every pair while keeping the overall false-positive rate at the α you chose.

Can I do a two-way ANOVA here?

No. This calculator does one-way ANOVA only, including Welch's unequal-variance version. A two-way design — two factors and their interaction — needs software with a model formula, such as R, Python's statsmodels, SPSS or JASP. Saying so is more useful than quietly computing something else.

What is the difference between one-way ANOVA and multiple t-tests?

The error rate. Each t-test at α = 0.05 has a 5% chance of a false positive; with three groups there are three pairs and the chance of at least one false positive rises to about 14%, with five groups it is 40%. ANOVA asks a single question at the level you set.

What is a good F value?

F has no meaning without its two degrees of freedom. On 2 and 12 df, F above 3.89 is significant at 0.05; on 2 and 100 df it takes only 3.09. Report F with both df and the p-value: F(2, 12) = 18.67, p < 0.001.

My groups have different sample sizes. Is that a problem?

Not in itself — ANOVA handles unequal group sizes. It becomes a problem when the variances also differ, because the classic F is then noticeably wrong in a direction that depends on whether the larger group has the larger variance. Use the Welch version, which is reported for every calculation above.