Gaussian Stein Identity

The Gaussian Stein Identity is a powerful theorem in probability and statistics that relates the expectation of a function multiplied by a centered Gaussian random variable to the expectation of the function's derivative. It provides a way to transform complex expectations into simpler derivative-based forms, which is extensively used in Stein's method for normal approximation, constructing score functions in machine learning, and exact calculations in stochastic calculus.

Worked Example

Let X be a normally distributed random variable, X ~ N(μ, σ²). We want to find the expectation E[(X - μ)f(X)] where f(X) is a differentiable function, specifically f(x) = x².

By the Gaussian Stein Identity, this is equivalent to σ² E[f'(X)]. Let's calculate both sides independently for a standard normal distribution (μ = 0, σ = 1).

Left Hand Side (Centered Product Expectation):
We compute E[(X - 0) * X²] = E[X³].
For a standard normal distribution, all odd moments are exactly 0. Thus, E[X³] = 0.
Right Hand Side (Variance times Expected Derivative):
The derivative of f(x) = x² is f'(x) = 2x.
We need to compute 1² * E[2X] = 2 * E[X].
Since the mean of a standard normal is 0, this equals 2 * 0 = 0.

Both sides independently evaluate to 0, confirming the identity holds.

Experiment

Explore the Gaussian Stein Identity interactively in the Gaussian Stein Identity module.

Set the Mean to 1 and the Standard deviation to 1.
Select the Linear function. Observe that E[f'(X)] is exactly 1, since the derivative of x is 1. The centered product expectation is 1, and since variance is 1, both sides match.
Change the Function to Quadratic. Now E[f'(X)] becomes 2, because f'(x) = 2x and the mean is 1. The identity holds with both sides evaluating to 2.
Change the Function to Exponential and adjust the Exponent t. The interactive chart will immediately display the density-weighted integrands, and you can see that the identity residual always remains at exactly 0.

Limitations

The Gaussian Stein Identity relies on a few core assumptions that limit its universal application:

The following primary source contains the original derivation and in-depth discussion of this identity.

Differentiability: The function f(x) must be absolutely continuous (almost everywhere differentiable), and the expectation E[|f'(X)|] must exist and be finite.
Gaussian Assumption: The identity in its simplest form strictly assumes that X follows a Gaussian (normal) distribution. For other distributions, the identity must be generalized using a different density-specific operator (a Stein operator).
Degenerate Point Mass: When the standard deviation (σ) is 0, the distribution collapses to a degenerate point mass at the mean (μ). The probability density function is no longer defined everywhere, and both sides of the identity collapse to 0. This boundary case is explicitly supported in the interactive module visualization.

References

The following primary source contains the original derivation and in-depth discussion of this identity.

Charles Stein, "Approximate Computation of Expectations" (Chapter 2) - Foundational introduction to the method, discussing the fundamental lemma characterizing the normal distribution via this identity.

Gaussian Stein Identity