Holm Step-Down Procedure

Calculations and Stopping Rule

Unlike the Benjamini-Hochberg procedure, which focuses on the proportion of false discoveries and operates as a step-up rule, Holm's method is a step-down rule. Given 4 p-values: 3/50 (0.06), 1/100 (0.01), 1/5 (0.2), and 1/25 (0.04) with family α = 0.10:

1. Sort the p-values from smallest to largest: 0.01, 0.04, 0.06, 0.2.
2. For the smallest (rank 1), compare to 0.10 / (4 - 1 + 1) = 0.025. Since 0.01 ≤ 0.025, it is rejected.
3. For rank 2, compare to 0.10 / (4 - 2 + 1) = 0.0333. Since 0.04 > 0.0333, it fails.
4. The procedure permanently stops at the first failure. Rank 2 and all subsequent ranks are retained.

First compute the adjusted value at each sorted rank using min(1, max_j≤i ((4 - j + 1) * p_j)), then map those values back to the original hypothesis IDs. The resulting adjusted values are 0.12, 0.04, 0.2, and 0.12 respectively.

Interactive Demonstration

Open the interactive experiment to visualize how the thresholds change for different p-value families and targets.

1. Select the A family and α = 0.10. Observe the first failure stops the procedure.
2. Select the C family and α = 0.05. Notice how all hypotheses can pass if they are sufficiently small, reflecting a sequential Bonferroni-style correction that gets less stringent.
3. Select Threshold equality. Equality rejects only while the step-down procedure remains active: with Family alpha = 0.05, ID1 rejects, rank 2 fails, and ID4 remains retained even though it equals its own later threshold.

Assumptions and Boundaries

The procedure provides strong FWER control for any valid individual p-values under arbitrary dependence structures. However, it is generally more conservative than procedures focusing solely on false discovery rates. The arithmetic comparisons use exact rational fractions; six-decimal representations are only for display. Equality logic holds precisely for analytically exact endpoints (e.g., zero).

Primary References

This method was originally introduced by Sture Holm (1979) as a simple, sequentially rejective multiple test procedure. See the official R documentation for related implementations: p.adjust.