McNemar Exact Test
Track paired binary changes and enumerate the discordant outcomes behind an exact McNemar p-value.
By Riano · Published · Updated · Report a correction
McNemar Exact Test
Concept Overview
The McNemar Exact Test evaluates paired binary responses, such as evaluating successes before and after a treatment. Rather than modeling all outcomes, the exact test conditions only on the discordant pairs—the cases that changed state. Under the null hypothesis of marginal homogeneity, a discordant pair is equally likely to be a loss or a gain, leading to a binomial distribution for the test statistic.
Independent Calculation Example
Consider a hypothetical scenario with matched binary outcomes (e.g., passing a test before and after a tutoring session). Suppose we have fixed concordant counts of 4 and 4. We observe Losses b = 2 and Gains c = 4.
The total discordant count is:
n = b + c = 2 + 4 = 6
The exact test uses a Binomial distribution B ~ Binomial(n, 1/2). The test statistic is min(b, c), which is min(2, 4) = 2.
The one-sided probability of observing 2 or fewer outcomes is:
P(B ≤ 2) = P(B = 0) + P(B = 1) + P(B = 2) P(B ≤ 2) = (1 + 6 + 15) / 2^6 = 22 / 64 = 11 / 32
To obtain the two-sided p-value, we double the one-sided tail (capped at 1):
| Metric | Value |
|---|---|
| Discordant n | 6 |
| Statistic min(b, c) | 2 |
| Two-sided p-value | 22/32 (0.6875) |
We observe a p-value of 0.6875, which fails to reject the null hypothesis of marginal homogeneity at typical significance levels.
Interactive Experiment
In the interactive module, you can directly adjust the number of losses and gains in a paired experiment setting. The visualizer graphs the probability mass function for the exact conditional binomial distribution.
- Adjust the Losses b slider to change the count of pairs transitioning from success to failure.
- Adjust the Gains c slider to change the count of pairs transitioning from failure to success.
- Observe the dynamically updated Statistic min(b, c) and exact conditional Exact Two-Sided p-value based on the specified pair counts.
Try it out in the matching experiment: https://riano.app/module/mcnemar-exact-test.
Assumptions and Limitations
The McNemar Exact Test assumes that the outcomes of the pairs are mutually independent. Additionally, it inherently drops concordant pairs (those that do not change state), which might contain useful information about total prevalence or test-retest reliability, despite not contributing to the assessment of marginal changes. Furthermore, doubling the one-sided p-value guarantees a valid test but might lead to conservative inferences due to the discrete nature of the binomial distribution.
References
The implementation and exact interpretation follow the guidelines of the statsmodels official mcnemar documentation, detailing the exact binomial test condition on the discordant-count minimum statistic.
Experience it interactively
Adjust parameters, observe in real time, and build deep intuition with Riano’s interactive McNemar Exact Test module.
Try McNemar Exact Test on Riano →