Paley–Zygmund Lower Bound

The Paley–Zygmund inequality is a fundamental result in probability theory that guarantees a lower bound on the probability that a nonnegative random variable is bounded away from zero. It states that for any nonnegative random variable X with finite variance, and for any constant 0 ≤ θ ≤ 1, the probability that X exceeds a fraction θ of its expected value is bounded below by a term involving its first and second moments.

Second-Moment Argument

The bound is derived using the Cauchy-Schwarz inequality. We write the expectation E[X] by splitting X based on the event X > θE[X]:
E[X] = E[X I(X ≤ θE[X])] + E[X I(X > θE[X])]
Since X is nonnegative, E[X I(X ≤ θE[X])] ≤ θE[X]. Therefore:
E[X] ≤ θE[X] + E[X I(X > θE[X])]
(1 - θ)E[X] ≤ E[X I(X > θE[X])]
Applying Cauchy-Schwarz to E[X I(X > θE[X])]:
E[X I(X > θE[X])]² ≤ E[X²] E[I(X > θE[X])²] = E[X²] P(X > θE[X])
Combining these gives the Paley–Zygmund lower bound:
P(X > θE[X]) ≥ (1 - θ)² E[X]² / E[X²]

Let's consider an example with a Binomial distribution, X ~ Binomial(n=4, p=0.5).

X ~ Binomial(n=4, p=0.5) E[X] = np = 4 * 0.5 = 2.0 Var(X) = np(1-p) = 4 * 0.5 * 0.5 = 1.0 E[X²] = Var(X) + E[X]² = 1.0 + 4.0 = 5.0 Let θ = 0.5. The threshold is θE[X] = 0.5 * 2.0 = 1.0. Calculate the exact strict tail P(X > 1): P(X = 2) = 6 * (0.5)^4 = 0.375 P(X = 3) = 4 * (0.5)^4 = 0.25 P(X = 4) = 1 * (0.5)^4 = 0.0625 P(X > 1) = 0.375 + 0.25 + 0.0625 = 0.6875 Calculate the Paley-Zygmund lower bound: P(X > 1) ≥ (1 - 0.5)² * (2.0)² / 5.0 = 0.25 * 4.0 / 5.0 = 0.2 The bound (0.2) is indeed lower than the exact probability (0.6875).

Experiment

This experiment demonstrates how to use the interactive Paley-Zygmund Lower Bound module to investigate the relationship between the actual strict tail probability and its lower bound.

Open the Paley–Zygmund Lower Bound module.
Set Trials to 4 and Success p to 0.5.
Set Threshold theta to 0.5. Observe the threshold marked on the graph.
The exact tail probability is 0.6875, and the Paley-Zygmund lower bound is 0.2. Note the bound slack of 0.4875.
Change Threshold theta to 1. The lower bound becomes 0, showing that for θ=1 the bound evaluates trivially.

Limitations

The Paley-Zygmund bound requires the random variable X to be nonnegative. If X can take negative values, the inequality does not hold. It is a lower bound; it does not approximate or upper-bound the tail. When the variance is very large compared to the squared expectation, the bound can be quite loose (close to zero) and sometimes trivial. For θ = 1, the bound is exactly zero, making it uninformative. If the variable is identically zero (e.g. Binomial with p=0), the bound becomes trivially zero to avoid division by zero.

References

For further reading and a detailed mathematical sketch of the Paley-Zygmund Inequality and related bounds, please refer to the following resource:

Stanford Theory - Paley-Zygmund Inequality Sketch

Paley–Zygmund Lower Bound