Law of Total Variance

Worked Example

Let G be a group indicator where G = + with probability p = 0.25, and G = - with probability 1-p = 0.75. Suppose the conditional mean for G = + is +d = +2, and for G = - is -d = -2. Within each group, X varies by a fair sign adding +s or -s, where s = 1.

1. The mean of X is E[X] = p(d) + (1-p)(-d) = 0.25(2) + 0.75(-2) = 0.5 - 1.5 = -1.
2. Within-group variance is E[Var(X|G)] = s² = 1² = 1.
3. Between-group variance is Var(E[X|G]) = 4p(1-p)d² = 4(0.25)(0.75)(2²) = 0.75 * 4 = 3.
4. Directly enumerate the four atoms: X = 3, 1, −1, −3 with probabilities 1/8, 1/8, 3/8, 3/8. Relative to mean −1, the weighted squared deviations are 2, 1/2, 0, 3/2. Their independently computed sum is 4.
5. The separate decomposition sum is 1 + 3 = 4. Direct total minus decomposition is zero; neither side is filled from the other.

Experiment

You can verify this decomposition using the interactive Law of Total Variance.

1. Set "Group-plus probability p" to 0.25.
2. Set "Mean magnitude d" to 2.
3. Set "Within-group spread s" to 1.
4. Observe that the Within Variance is 1.000000.
5. Observe that the Between Variance is 3.000000.
6. Read all four atom values/probabilities and their weighted variance terms. The Direct Total Variance and separate decomposition sum are both 4.000000.
7. Set Group-plus probability p to 0. The plus branch has zero mass and is marked unused. The mean is −2, within variance 1, between variance 0 and direct total 1.
8. Set Group-plus probability p to 0.5, Mean magnitude d to 0 and Within-group spread s to 0. All atoms coincide at zero; their combined mass is 1 and every variance is zero.
9. Set Group-plus probability p to 0.75, Mean magnitude d to 2 and Within-group spread s to 0.5. Mean is 1, within variance 1/4, between variance 3 and direct total 13/4.

Limitations and Assumptions

This model assumes fixed spreads (s) within both groups for simplicity. In real data, conditional variances often differ between groups (heteroscedasticity). Furthermore, a group with zero probability (p=0 or p=1) simply acts as an unused branch of the mixture model, meaning the conditional variance and conditional mean exist as parameters of the unused branch, but do not contribute to the overall expectations. It is not equivalent to dividing by zero: conditional distributions on a null group are not uniquely determined by the joint law. This forward model stores an unused branch parameter. The four latent branch/sign atoms remain visible even when values coincide; their probabilities add in the marginal chart. Calculations use exact rational arithmetic on the supported control grid, with rounding only in six-place display fields. The decomposition needs finite second moments; these bounded four-atom distributions satisfy that requirement.

References

The standard theory of conditional expectation and total variance decomposition can be found in probability and statistics literature. A clear presentation of the decomposition is provided in the lecture notes below.

• Lecture 4: Conditional Probability and Expectation - Yen-Chi Chen, University of Washington