Bonferroni and Šidák Corrections

When performing multiple statistical tests, the chance of making at least one Type I error (a false positive) increases. To control the Family-Wise Error Rate (FWER)—the probability of one or more false discoveries—we must require a stricter threshold for each individual test. The Bonferroni and Šidák methods provide two classical ways to calculate this per-test threshold.

Worked Example

Suppose we are conducting m = 5 tests, and we wish to bound our overall FWER at α = 0.05.

Bonferroni Correction: Uses a simple union bound. The per-test threshold is exactly α/m.
Calculation: 0.05 / 5 = 0.01.
Expected FWER: 1 - (1 - 0.01)⁵ = 0.049009950100000005.
Šidák Correction: Assumes the p-values of the tests are statistically independent and uniformly distributed under the null hypothesis. The per-test threshold is exactly 1 - (1 - α)^1/m.
Calculation: 1 - (1 - 0.05)^1/5 = 0.010206218313011496.
Expected FWER: 1 - (1 - 0.010206218313011496)⁵ = 0.05.

These exact calculations show that Šidák is exact under independence, while Bonferroni gives a stricter cutoff that algebraically bounds the FWER.

Experiment

In the interactive module, you can investigate how changing parameters affects these error rates.

Adjust the Test count (m) control to 20 and the target Family alpha (α) to 0.05.
Observe the calculated threshold spans. The Šidák threshold exactly hits the expected line on the visual plot, and the Bonferroni threshold falls slightly lower.

Notice that for m=1, the two thresholds are mathematically identical.

Limitations

Both the Bonferroni and Šidák corrections control the Family-Wise Error Rate (FWER), which is very strict. In modern high-dimensional settings with thousands of tests (like genomics), these single-step FWER procedures often lead to unacceptably low power, rejecting very few true effects. In such cases, controlling the False Discovery Rate (FDR)—for instance, via the Benjamini-Hochberg procedure—is often preferred.

Furthermore, while the Šidák correction is exact under complete independence, it may fail to control the FWER if the tests are positively or negatively correlated. Bonferroni, though slightly more conservative, safely controls the FWER regardless of how the tests depend on each other.

References

Statsmodels Multi-test documentation: statsmodels.stats.multitest.multipletests

The implementation in libraries such as statsmodels provides these threshold and p-value correction methods for practical data analysis tasks.

Bonferroni and Šidák Corrections