Hypothesis tests
McNemar's Test Calculator
Test whether a yes/no outcome changed, when the same subjects — or matched pairs — are measured twice.
Also called mcnemar test calculator, McNemar chi-square calculator.
Have a spreadsheet? Drop a CSV here
Or . CSV, TSV or plain text. You pick which column is which after it loads — the names in your file do not have to match ours. The file is read in your browser and never uploaded.
How to read this result
Only the discordant pairs count. The subjects who answered the same way twice — the a and d cells — carry no information about which direction the change goes, so the test ignores them entirely. That is not a simplification; it is the whole idea of a paired design.
Under the null hypothesis, each discordant pair is a coin flip. If the treatment changes nothing, a pair that changed is equally likely to have changed in either direction. So the test asks a single question: is b versus c a plausible split of b + c fair coin flips?
With few discordant pairs, use the exact binomial p-value. The χ² form is an approximation to the same binomial, and below about 25 discordant pairs it is not a good one. The calculator shows both and puts the exact one first when the count is small.
A significant result means the change is systematic, not that it is large. The proportion before and after, and the difference between them, are the numbers that describe the size.
The formula
- pairs that were yes then no
- pairs that were no then yes
One degree of freedom. The −1 is the continuity correction; without it the statistic is (b − c)²/(b + c). The exact alternative treats b as a draw from Binomial(b + c, ½).
Worked example
Did a debate change how people intend to vote?
314 people are asked before and after a televised debate whether they approve of a policy. 101 approved both times and 33 disapproved both times — those 134 tell us nothing about change. The interesting cells are the 121 who switched from approve to disapprove and the 59 who switched the other way.
| after: yes | after: no | |
|---|---|---|
| before: yes | 101 | 121 |
| before: no | 59 | 33 |
- Identify the discordant pairs.b = 121 switched away, c = 59 switched toward
- Under the null hypothesis those 180 switches are 180 fair coin flips, so we would expect about 90 each way.expected 90 and 90
- Compute χ² with the continuity correction.(|121 − 59| − 1)² / 180 = 61² / 180 = 3721 / 180 = 20.67
- One degree of freedom, upper tail.p = 0.0000055
- The exact binomial version: 2 × P(X ≤ 59) with X ~ Binomial(180, ½).p = 0.0000044
Approval fell from 70.7% to 50.9%. Analysing the same numbers as two independent samples with a chi-square test would throw away the pairing and produce a much weaker result — the paired design is what makes 314 people enough.
Checked against R's mcnemar.test.
The calculator above is loaded with these numbers by the Load the worked example button.
Assumptions, and when to use something else
- The pairs are genuinely paired. Same subject twice, or matched on the variables that matter.
- The outcome is binary. For an ordered outcome with more than two levels, the Stuart-Maxwell or Bowker extensions apply; they are not implemented here.
- Pairs are independent of each other, even though the two measurements within a pair are not.
- Enough discordant pairs for the χ² form — below about 25, read the exact binomial p-value, which the calculator computes regardless.
- The two groups are different peopleChi-square testIndependent groups, not pairs — the ordinary test of independence applies.
- The paired outcome is a measurement, not yes/noPaired t-testFor a continuous outcome measured twice, the paired t-test is the equivalent.
- Paired measurements that are not normally distributedWilcoxon signed-rank testThe rank-based paired test, for ordinal or skewed measurements.
Questions people ask
Why does McNemar's test ignore the agreeing cells?
Because they contain no information about the direction of change. A person who said yes both times is consistent with any amount of change in either direction among the people who did switch. Conditioning on the discordant pairs is what makes the test exact and what makes a paired design powerful.
What if I have very few discordant pairs?
Read the exact binomial p-value, which the calculator always reports. With fewer than about 25 discordant pairs the χ² approximation is unreliable, and with fewer than 10 it is best ignored entirely.
Can I use McNemar's test for inter-rater agreement?
It tests whether two raters differ systematically — whether one says yes more often than the other. It does not measure agreement: two raters can pass this test while disagreeing constantly, as long as they disagree symmetrically. For agreement, use Cohen's kappa.
Should the continuity correction be on or off?
R applies it by default, and it makes the test more conservative. When the number of discordant pairs is small enough for the correction to matter noticeably, the exact binomial test is the better answer than either version.