Probability Generating Functions

In probability theory, the probability generating function (PGF) is a useful tool for dealing with discrete random variables that take non-negative integer values. By representing the probabilities of a random variable as the coefficients of a polynomial (or power series), we can use algebraic operations like differentiation and polynomial multiplication to derive moments and distributions of sums.

Concept Overview

For a discrete random variable X taking non-negative integer values, its probability generating function G(z) is defined as the expected value of z^X:

G(z) = E[z^X] = ∑ p_k z^k

where p_k = P(X = k). The variable z is a dummy variable (or "probe"), and the probabilities p_k are the coefficients of the polynomial.

Worked Example

Consider the sum of two independent fair coins (Bernoulli trials). Let X be the number of heads. We can trace the calculation step by step to find the exact expected probabilities of the sum.

The PGF for a single coin flip is G₁(z) = P(X=0)z⁰ + P(X=1)z¹ = 0.5 + 0.5z.
The sum of two independent copies has a PGF equal to the square of the individual PGF: G₂(z) = (0.5 + 0.5z)².
Expanding the polynomial gives: G₂(z) = 0.25 + 0.5z + 0.25z².
Evaluating at z=0.5 yields G₂(0.5) = 0.25 + 0.5(0.5) + 0.25(0.25) = 0.25 + 0.25 + 0.0625 = 0.5625.
The mean is found by evaluating the first derivative at z = 1: G'(z) = 0.5 + 0.5z. Evaluating at z=1 gives 1.0. For the sum, it is 2 * 0.5 = 1.0.

From the coefficients, we expect the zero-sum probability to be exactly 0.25, the PGF at z=0.5 to be 0.5625, and the mean to be 1.0.

Experiment

Use the Probability Generating Functions module to visualize how generating functions multiply when adding independent random variables.

Set the Distribution to Die.
Set Independent copies to 2. The canvas visualizes the probabilities of the sum of two dice (from 2 to 12).
Notice the triangular shape of the probabilities, which results from the polynomial convolution (multiplication) of the uniform distributions.
Change the Probe z to 0.5. Observe the computed PGF of sum, which evaluates the combined polynomial at z = 0.5.
Set Independent copies to 0. The polynomial becomes exactly G(z) = 1 (an empty sum is identically zero with probability 1).

Limitations and Boundaries

There are several notable theoretical limitations and boundaries to consider when applying the Probability Generating Functions approach. Let's analyze the most common scenarios where generating functions may fall short or require alternative mathematical formulations:

Domain of Application: PGFs are specifically designed for discrete random variables taking non-negative integer values. For continuous variables or variables that can take negative values, the Moment Generating Function (MGF) or Characteristic Function is required.
Finite representation: In this interactive module, numerical and visual calculations are limited to finite bounded distributions. Distributions with infinite support (like Poisson or Geometric) have generating functions that are infinite series, which are evaluated algebraically or through closed-form solutions rather than direct polynomial array convolution.
Evaluation at zero: When calculating z⁰ in computational environments, we explicitly define 0⁰ = 1 in this polynomial context to ensure the constant term (the zero-mass probability p₀) is correctly preserved when evaluating at the origin z=0.

References

Random Services: Generating Functions

This reference provides a comprehensive mathematical introduction to generating functions, including probability generating functions, their properties, and how they relate to expected values and sums of independent variables.

Probability Generating Functions