Benjamini-Hochberg Procedure

Find the largest passing rank

For Default, the sorted values are 0.01, 0.04, 0.041, 0.2 and 0.8. With q = 0.1 and m = 5, compare each rank i to i·q/5:

1. Rank 1: 0.01 ≤ 0.02.
2. Rank 2: 0.04 = 0.04; equality counts.
3. Rank 3: 0.041 ≤ 0.06.
4. Ranks 4 and 5 fail: 0.2 > 0.08 and 0.8 > 0.10.

The largest passing rank is k = 3. Reject every rank up to k, giving IDs 1, 2 and 3. BH is a step-up rule: it does not require every earlier rank to pass its own threshold. Bonferroni instead compares every p-value to q/5 = 0.02 and rejects only ID 1.

BH-adjusted p-values use the reverse cumulative minimum of 5p_j/j, capped at one. Rank 3 gives (5/3)·0.041 = 41/600 = 0.068333; rank 2 inherits the same lower adjusted value. The complete vector is 1/20, 41/600, 41/600, 1/4, 4/5.

Compare targets, ties and zero

Open the Benjamini-Hochberg experiment. The control q is a probability from zero to 0.20; for example 0.05 means a five-percent target.

1. Set P-value set to Default, Target q to 0.05 and Method to BH. Only ID 1 is rejected.
2. Set q to 0.10. IDs 1, 2 and 3 are rejected. Switch Method to Bonferroni: only ID 1 remains rejected.
3. Select Ties, q = 0.04 and BH. Rank 3 passes although ranks 1 and 2 fail their own thresholds; all three tied IDs are rejected. Each adjusted p-value is 1/30.
4. Select Zero and q = 0. Equality at zero rejects ID 1. Select Default while keeping q = 0: no IDs are rejected.

An expectation over repeated families

FDR is E[V/max(R,1)], with V false rejections and R total rejections. A nominal q does not guarantee V/R ≤ q in one list. This experiment declares no ground-truth labels and estimates no empirical FDR from its fixed p-values.

Valid null p-values are super-uniform: P(p ≤ u) ≤ u. Uniform values are a special case; conservative valid values need not be exactly uniform. BH's usual guarantee requires independence or the stated positive regression dependence, not arbitrary dependence. Bonferroni controls the chance of any false rejection under valid null p-values, using a different error criterion. The Zero set illustrates algorithmic equality, not a simulated valid-null distribution.

Sorting keeps original ID order for ties. Arithmetic comparisons use exact rational cross-products; six-decimal displays are rounded. The adjusted-p cap at one is part of the definition.

Definitions and supported assumptions

The official SciPy false_discovery_control reference documents adjusted p-values and the independence/positive-dependence conditions. Columbia Public Health's FDR guide explains the distinction between FDR and family-wise error. The five-value examples here are independently constructed.