Fisher's Exact Test

Concept Overview

Fisher's Exact Test examines whether the rows and columns in a 2×2 contingency table are independent. By conditioning on the observed row and column margins, the test considers only tables that preserve these totals. This turns a test of independence into a combinatorial counting problem: what is the chance of seeing a table as extreme as the one observed, among all tables with the same margins?

Condition on the observed margins

Consider a 2×2 table with cells a, b, c, and d. Let a = 1, b = 9, c = 11, and d = 3.

	Col 1	Col 2	Total (Row)
Row 1	a = 1	b = 9	10
Row 2	c = 11	d = 3	14
Total (Col)	12	12	24 = N

The margins are fixed: Row 1 total is 10, Col 1 total is 12, and the overall total N is 24. The possible values for the top-left cell x are bounded between 0 and 10.

Under the hypergeometric distribution, the exact probability of observing x in the top-left cell is:

P(X = x) = C(Col1, x) × C(Col2, Row1 - x) / C(N, Row1)

For x = 1: C(12, 1) × C(12, 9) = 12 × 220 = 2640.
Total combinations: C(24, 10) = 1961256.
P(X = 1) = 2640 / 1961256 = 10 / 7429 ≈ 0.001346

To compute the two-sided p-value using the probability-ordering convention, we sum the probabilities of all possible tables that are less than or equal to the probability of the observed table. By comparing exact integer numerators, we safely sum:

P(x = 0): numerator is 1 × 66 = 66 (≤ 2640)
P(x = 1): numerator is 2640 (≤ 2640)
P(x = 9): numerator is 220 × 12 = 2640 (≤ 2640)
P(x = 10): numerator is 66 × 1 = 66 (≤ 2640)

Summing these exact fractions gives a two-sided p-value of 41/14858 ≈ 0.002759. The exact odds ratio is (1×3)/(9×11) = 3/99 = 1/33 ≈ 0.030303.

Experimenting with Constraints

The Fisher's Exact Test explorerallows you to interactively manipulate the four cells of the 2×2 table.

Start with the default table: set Cell a to 1, Cell b to 9, Cell c to 11, and Cell d to 3. Observe the probability mass chart. The red dashed line marks the observed table, and bars colored teal represent the tables included in the two-sided p-value.
Change Cell a to 0, Cell b to 2, Cell c to 2, and Cell d to 0. The support narrows significantly to just [0, 2], and the two-sided p-value jumps to 1/3 ≈ 0.333333, with an odds ratio of exactly 0.
Set all cells to 1. The margins are completely symmetrical, resulting in a support of [0, 2], an odds ratio of 1, and a maximum possible two-sided p-value of 1.
Set all four cells to 0. There are no observations, so the p-value and odds ratio are undefined.
Keep a and b zero and set c and d to 1. The support is the singleton [0,0], both table probability and p-value are one, while the odds ratio remains undefined.

Limitations of the Test

Understanding these mathematical boundaries prevents misusing the test.

Fixed Margins Assumption: This null reference distribution conditions on both observed margins. They need not both have been fixed by the experimental design, but the sampling design and the conditional question still matter. A p-value alone does not establish causality or prove independence.
Conservatism: Because the hypergeometric distribution is discrete, the achievable p-values are sparse. The test can therefore be conservative: the achievable rejection probability may fall below a chosen nominal level. It need not be strictly lower at every possible level.
Two-Sided Alternatives: There is no single universal definition for a two-sided p-value in asymmetric discrete distributions. This module uses the probability-ordering method (summing all tables with probabilities ≤ the observed table), avoiding the crude "twice the one-sided tail" approach.

References

The following resources provide detailed explanations and standard algorithmic implementations for this test.

SciPy documentation: fisher_exact — Implementation details on computing the two-sided p-value via probability sums.

Fisher's Exact Test