Ville Anytime E-Process

In sequential hypothesis testing, we often want to track the evidence against a null hypothesis as data arrives, without being constrained to a fixed sample size. An e-process is a sequence of non-negative random variables where the expectation under the null hypothesis is always bounded by 1, making it a test martingale. Ville's inequality allows us to use this e-process to safely stop the experiment early if sufficient evidence is observed, maintaining valid error guarantees at any stopping time.

Worked Example

Suppose we want to test whether a coin is fair (the null hypothesis, where the probability of heads is p_0 = 0.5). We specify a fixed alternative hypothesis that the coin is biased towards heads, with q = 0.75. We observe a sequence of coin flips. Our e-process E_t starts at E_0 = 1.

The update for each flip is the likelihood ratio between the alternative and the null. For a head, the multiplier is q/(p_0) = 0.75/0.5 = 1.5. For a tail, the multiplier is (1-q)/(1-p_0) = 0.25/0.5 = 0.5.

Let's trace the e-value process for an "All heads" path over a horizon of N = 4 flips:

Start: t=0, E_0 = 1
Flip 1 (Head): E_1 = E_0 × 1.5 = 1 × 1.5 = 1.5
Flip 2 (Head): E_2 = E_1 × 1.5 = 1.5 × 1.5 = 2.25
Flip 3 (Head): E_3 = E_2 × 1.5 = 2.25 × 1.5 = 3.375
Flip 4 (Head): E_4 = E_3 × 1.5 = 3.375 × 1.5 = 5.0625

If our test level is α = 0.25, our evidence threshold for rejection is 1/α = 4. We reject the null hypothesis at the first time t where E_t ≥ 4. In this path, E_3 = 3.375 < 4, but E_4 = 5.0625 ≥ 4. Thus, the first crossing time is t = 4. The exact null probability of ever crossing this threshold within 4 steps is the fraction of all 2^4 = 16 possible null paths that reach an e-value ≥ 4, which we can compute by enumerating all paths.

Experiment

You can interactively explore this e-process in the Ville Anytime E-Process module. Follow these steps to observe Ville's inequality in action:

Set the Horizon to 6.
Set the Alternative probability to 0.75.
Set the Test level to 0.5.
Observe the Observed path set to All heads.
Note that the evidence threshold is 1/0.5 = 2.0. The e-process trace crosses this threshold at t = 2.
Change the Observed path to Alternating. The trace will go up on the first flip, down on the second, and so on. It never crosses the threshold.
Regardless of the chosen path, observe the Exact null crossing probability. This value is computed by examining all 2^N possible paths and remains valid for the chosen horizon, test level, and alternative.

Limitations

This module assumes independent and identically distributed (iid) Bernoulli trials under a simple null hypothesis (p_0 = 0.5) against a fixed simple alternative (q). The e-value construction directly depends on this exact alternative. If the true data-generating process differs, the specific power characteristics will change, though the e-process remains a valid null martingale as long as the null distribution is correct.

Furthermore, Ville's inequality provides a conservative upper bound. The actual probability of crossing the threshold in a finite horizon is often strictly less than the theoretical upper bound α. In this module, the exact null crossing probability is computed via exhaustive enumeration up to N=8, which demonstrates this conservatism.

Finally, an e-value should not be interpreted as a calibrated posterior probability of the null hypothesis. It is purely a measure of evidence constructed as a likelihood ratio against the null.

References

The theoretical foundations of e-values and their use in anytime-valid sequential inference are extensively discussed in the literature. For a comprehensive overview of testing by betting, e-processes, and Ville's inequality, refer to:

Ramdas, A., et al. (2020). "Testing by betting: A strategy for statistical and scientific communication".

Ville Anytime E-Process