James–Stein Positive-Part Shrinkage

Worked Example

Let us trace an independently calculated example for a single deterministic observation. Suppose the dimension is d = 5, the noise standard deviation is sigma = 1, and the true mean vector is exactly the origin (mu1 = 0, with other coordinates also 0). The observed realization Y happens to lie at distance r = 2 sigma along the first axis, so Y = (2, 0, 0, 0, 0).

1. Calculate the squared norm of the observation: ||Y||² = r² × sigma² = 2² × 1² = 4.
2. Evaluate the ordinary James-Stein shrinkage factor: 1 - (d - 2) * sigma² / ||Y||² = 1 - (5 - 2) * 1 / 4 = 1 - 3/4 = 0.25.
3. Calculate the ordinary first estimate: 0.25 × 2 = 0.5.
4. Determine the positive-part factor by taking the maximum of 0 and the ordinary factor: max(0, 0.25) = 0.25.
5. Calculate the positive-part first estimate: 0.25 × 2 = 0.5.
6. Compare the realized squared errors for this particular observation against the truth (0, 0, 0, 0, 0). The MLE Y has a squared error of (2 - 0)² = 4.0. Both the ordinary and positive-part estimates have a squared error of (0.5 - 0)² = 0.25. For this observation, shrinkage substantially reduces the realized error.

Experiment

You can interactively observe how the observed radius affects the shrinkage factor and the estimates. Open the interactive module at https://riano.app/module/james-stein-positive-part-shrinkage and try the following steps:

1. Set Dimension to 5 and Noise sigma to 1.
2. Set Observed radius in sigma to 1. Notice that the ordinary factor is 1 - 3/1 = -2.0. This negative factor pushes the ordinary estimate past the origin to -2.0, increasing the realized squared error to 4.0 compared to the MLE error of 1.0.
3. Observe that for the same inputs, the positive-part factor clamps to 0.0, yielding a positive-part estimate of 0.0. The realized error for the positive-part estimator is perfectly 0.0 here since it correctly stopped at the origin.
4. Change Observed radius in sigma to 0. Notice the ordinary formulas become "Unavailable" due to division by zero, but the positive-part factor extends continuously to 0.0.

Limitations

It is crucial to distinguish between realized error for a single specific observation and the expected risk integrated over the entire Gaussian distribution. The dominance theorem for the James-Stein estimator states that its expected risk is strictly less than the MLE risk everywhere. However, for a given, specifically chosen individual observation, the shrinkage estimator may actually incur a larger error than the MLE. Furthermore, the ordinary James-Stein estimator can yield a negative shrinkage factor that reverses the sign of the observation, a property the positive-part modification fixes. Note that this module does not estimate unknown sigma or provide Monte Carlo proofs, and the fundamental theorem does not hold for dimensions 1 or 2.

References

For a mathematical treatment of the James-Stein estimator and its expected-risk dominance properties, refer to the following lecture notes:

Statistics 210b Lecture Notes - James-Stein Estimator