Hypothesis tests

Tukey's HSD Calculator

After a significant ANOVA, find out which pairs of groups actually differ — with the false-positive rate held at α across all the comparisons, not per comparison.

Updated August 2026Runs in your browser — nothing is uploadedVerified against R

How to read this result

Every p-value in the table is already adjusted. Compare it directly with α; there is no further correction to apply. A pairwise t-test on the same two groups would give a smaller p-value, and that difference is precisely the inflation Tukey's test exists to remove.

The intervals are simultaneous. The confidence level applies to the whole set at once: with 95% family-wise confidence, there is a 95% chance that every interval in the table covers its true difference. That is a much stronger — and wider — statement than 95% confidence on each one separately.

Read the intervals, not just the stars. A pair whose interval runs from 0.2 to 8.5 differs significantly and is also almost completely unquantified. A pair whose interval runs from 5.8 to 6.2 is both significant and precise.

The critical q value is the threshold on the studentized range distribution — the distribution of the largest gap you would expect to see among k group means by chance alone. That is where the correction comes from: with more groups, the largest gap is naturally larger, so the bar rises.

The formula

q = \frac{| {\bar{x}}_{i} - {\bar{x}}_{j} |}{\sqrt{\frac{M S_{w i t h i n}}{2} (\frac{1}{n_{i}} + \frac{1}{n_{j}})}}

$M S_{w i t h i n}$: the within-group mean square from the ANOVA — the pooled variance
$n_{i}, n_{j}$: the sizes of the two groups being compared
$q$: the studentized range statistic for this pair

q is compared against the studentized range distribution with k groups and N − k degrees of freedom. With unequal group sizes this is the Tukey-Kramer variant, which is what R's TukeyHSD reports.

The simultaneous interval

({\bar{x}}_{i} - {\bar{x}}_{j}) \pm q_{α, k, ν} \sqrt{\frac{M S_{w i t h i n}}{2} (\frac{1}{n_{i}} + \frac{1}{n_{j}})}

With equal group sizes the half-width is the same for every pair, and is called the honestly significant difference — the HSD in the name.

Worked example

Which fertiliser actually differs?

The ANOVA example found F(2, 12) = 18.67, p = 0.0002 across a control and two fertilisers. That says something differs. Tukey's HSD says what.

Control: 23, 25, 21, 24, 22 · Fertiliser A: 28, 30, 27, 29, 31 · Fertiliser B: 25, 24, 26, 23, 27

Take the within-group mean square from the ANOVA — the pooled variance.
MSW = 2.5 on 12 degrees of freedom
Look up the critical studentized range for k = 3 groups and 12 df at α = 0.05.
q* = 3.7729
The standard error is the same for every pair here, because the groups are equal in size.
SE = √(2.5/2 × (1/5 + 1/5)) = √0.5 = 0.7071
The honestly significant difference is q* × SE — any gap larger than this is significant.
HSD = 3.7729 × 0.7071 = 2.668
Compare each pair's gap against it.
Control vs A: 6.0 > 2.67 ✓ · Control vs B: 2.0 < 2.67 ✗ · A vs B: 4.0 > 2.67 ✓

Control vs Fertiliser A differ, p = 0.00017Control vs Fertiliser B no difference, p = 0.155Fertiliser A vs Fertiliser B differ, p = 0.0046q critical 3.773

Fertiliser A is the only treatment that stands apart from the control. Fertiliser B's two-tonne advantage does not clear the bar, and the cost of the correction is visible in that comparison: tested on its own against the pooled variance it gives p = 0.069, and Tukey's adjustment moves it to 0.155. That gap is the price of having looked at all three pairs rather than one.

Checked against R's TukeyHSD(aov(...)).

The calculator above is loaded with these numbers by the Load the worked example button.

Assumptions, and when to use something else

Tukey's HSD inherits ANOVA's assumptions, because it is built on ANOVA's pooled variance: independent observations, roughly normal groups, and comparable variances. It additionally assumes you are comparing all pairs — that is what it corrects for.

Two practical consequences:

If you only ever cared about a few specific comparisons, planned in advance, Tukey is conservative and a planned contrast would be more powerful.
If the variances are clearly unequal, the pooled MS within is the wrong yardstick and the Games-Howell procedure is the appropriate alternative — it is not implemented here, and R's PMCMRplus or Python's pingouin will do it.

You have not run the ANOVA yetOne-way ANOVARun it first. If F is not significant, pairwise testing afterwards is fishing.
The data are not normalKruskal-Wallis testThe rank-based omnibus test; follow it with pairwise Mann-Whitney tests and a Bonferroni correction.
Only two groups to compareT-testWith one comparison there is nothing to correct for.

Questions people ask

What does HSD stand for?

Honestly significant difference. Tukey's point in 1949 was that the differences other procedures called significant were not honestly so, because they ignored how many comparisons had been made. With k groups you make k(k−1)/2 comparisons, and the largest gap among them is naturally bigger than any single pre-planned gap.

Do I need a significant ANOVA before running Tukey's HSD?

Traditionally yes, and it is still the convention. Tukey's test controls the family-wise error rate on its own, so a protective ANOVA is not mathematically required — but running post-hoc tests after a non-significant ANOVA and reporting whatever turns up is fishing, and it reads as fishing.

Why are the Tukey p-values larger than a t-test's?

Because they are adjusted for the number of comparisons. With three groups there are three chances to find a difference by luck; the adjustment raises the bar so that the chance of any false positive stays at 5%. The unadjusted p-value answers a question nobody asked: 'if this were the only comparison I had ever considered…'.

What about Bonferroni instead?

Bonferroni divides α by the number of comparisons and is simpler, more general and more conservative. For all-pairs comparisons after ANOVA, Tukey is the more powerful choice because it uses the actual distribution of the largest gap rather than a worst-case bound.