Gaussian Entropy Power and Independence
See why independent Gaussian sums saturate an entropy bound while correlated sums need not.
By Riano · Published · Updated · Report a correction
Gaussian Entropy Power and Independence
The entropy power of a random variable is the variance of a standard Gaussian that has the same differential entropy. For independent Gaussian variables, the entropy power of their sum is equal to the sum of their entropy powers, establishing an important equality condition for the Entropy Power Inequality (EPI). When variables are correlated, this relationship changes significantly.
Worked Example
We calculate the entropy power and independent bound for a standard deviation 1 and standard deviation 2 with correlation 0.
| Metric | Value |
|---|---|
| Variance of Sum | 5 |
| Differential Entropy | 2.223657489421723 |
| Entropy Power | 5 |
Experiment
- Set First standard deviation to 1, Second to 2, and Correlation to 0, observing the Entropy power sum matches the Independent entropy-power bound at 5.
- Change Correlation to 1 and observe the Entropy power sum increases to 9.
Notice that negative correlation can violate the numeric sum condition without breaking the independent bound. Open the Interactive Module
Limitations
This visualization assumes jointly Gaussian distributions. For arbitrary distributions, the entropy power inequality still holds but the exact entropy values will differ.
References
The entropy power inequality requires independence for equality. See further discussion in The Entropy Power Inequality.
Experience it interactively
Adjust parameters, observe in real time, and build deep intuition with Riano’s interactive Gaussian Entropy Power and Independence module.
Try Gaussian Entropy Power and Independence on Riano →