Le Cam Rare-Event Limit

The Poisson Approximation to the Binomial

When modeling the number of successes in a large number of independent trials, each with a small probability of success, the Binomial distribution can be well approximated by a Poisson distribution with the same mean. Le Cam's inequality provides a concrete bound on how large the discrepancy between these two distributions can be.

Consider n independent Bernoulli trials, each with success probability p = λ/n. The total number of successes X follows a Binomial(n, p) distribution, with mean λ. The Poisson distribution with mean λ represents the limiting behavior of X as n → ∞ while λ remains fixed.

We measure the distance between the two distributions using the Total Variation (TV) distance. By definition, the TV distance between two probability mass functions P and Q on the integers is half their L_1 distance:

TV(P, Q) = 1/2 \sum_(k=0)^∞ |P(k) - Q(k)|

An equivalent and very useful formula for TV distance is:

TV(P, Q) = 1 - \sum_(k=0)^∞ min(P(k), Q(k))

Le Cam (1960) proved that the TV distance between Binomial(n, p) and Poisson(np) is bounded by n p². For p = λ/n, this bound becomes λ²/n. This guarantees that if n is large and λ²/n is small, the approximation error is strictly controlled everywhere. Under the exact overlap convention, this bound is min(1, n p²).

A checked calculation for 4 trials

Let's trace the calculation for n = 4 and a fixed mean λ = 1. The Bernoulli success probability is p = 1/4 = 0.25.

The PMF for Binomial(4, 0.25) evaluated at counts k ∈ {0, 1, 2, 3, 4}:

P_B(0) = C(4, 0) (0.25)^0 (0.75)^4 = 0.31640625
P_B(1) = C(4, 1) (0.25)^1 (0.75)³ = 0.421875
P_B(2) = C(4, 2) (0.25)² (0.75)² ≈ 0.2109375
P_B(3) = C(4, 3) (0.25)³ (0.75)^1 = 0.046875
P_B(4) = C(4, 4) (0.25)^4 (0.75)^0 ≈ 0.00390625

The PMF for Poisson(1):

P_P(0) = (e⁻¹ 1^0)/(0!) ≈ 0.36787944
P_P(1) = (e⁻¹ 1^1)/(1!) ≈ 0.36787944
P_P(2) = (e⁻¹ 1²)/(2!) ≈ 0.18393972
P_P(3) = (e⁻¹ 1³)/(3!) ≈ 0.06131324
P_P(4) = (e⁻¹ 1^4)/(4!) ≈ 0.01532831

The sum of overlaps is \sum_(k=0)^4 min(P_B(k), P_P(k)) = min(0.3164..., 0.3678...) + … ≈ 0.316406 + 0.367879 + 0.183939 + 0.046875 + 0.003906 = 0.91900666. The Total Variation is 1 - 0.91900666 = 0.08099333.

The Le Cam bound for this case is min(1, 4 × 0.25²) = 0.25. As expected, the actual TV distance 0.08099333 ≤ 0.25.

Testing the convergence

The Le Cam Rare-Event Limit explorer plots the Binomial and Poisson distributions, enabling direct visual and numerical comparison.

Set Fixed Poisson mean to 1, and Trials to 4. Check the Total Variation which matches our 0.08099 calculation.
Increase Trials to 16. The Bernoulli probability drops to 0.0625, the Le Cam bound drops to 0.0625, and the true Total Variation distance tightens to 0.01789.
The visualization clearly shows the Binomial bars converging closely around the Poisson curve points.
Adjust the Count probe to compare exact pointwise probabilities between the models.

Important Limitations

The Poisson limit requires the individual success probabilities p = λ/n to be small and the number of trials n to be large. It applies to counts of rare, independent events. This is different from the Central Limit Theorem, which requires the variance np(1-p) to grow large, yielding a Gaussian limit. If p is fixed (e.g., p=0.5) while n → ∞, the mean λ = np grows rather than staying fixed, and a Normal approximation (not Poisson) becomes appropriate.

References

The Total Variation property and Poisson bounds follow classical probability theory. A rigorous introduction and Le Cam's inequality can be found in university lecture notes on limit theorems:

Poisson Approximation (Cambridge DPMMS)

Le Cam Rare-Event Limit