SURE Soft-Threshold Risk

Choose denoising strength with a risk estimate that does not require knowing the true signal.

SURE Soft-Threshold Risk

Stein's Unbiased Risk Estimate (SURE) provides a way to estimate the mean squared error of a nearly arbitrary nonlinear estimator without knowing the true underlying signal. This is particularly powerful for tuning the threshold parameter in soft-thresholding denoising.

Learning Objectives

With observations Y = θ + σZ, where Z is standard independent Gaussian noise, we aim to derive the SURE formula for soft thresholding. The unbiased estimate of risk is given by:

SURE = ||η_λ(Y) - Y||² + 2σ² div η_λ(Y) - dσ²

At |y_i| = λ, we choose the derivative to be 0; the weak derivative is defined almost everywhere, so individual equality conventions do not affect the unbiasedness of the risk estimate.

Worked Example

Let's examine a specific sparse signal example to see how the SURE risk estimate is calculated and compare it to the true realized squared error.

Suppose the true signal is θ = [0, 0, 2, -2], and we observeY = [-0.3, 0.2, 1.8, -2.4]. We apply soft thresholding with λ = 0.5 and noise σ = 0.5.

True Signal (θ)     = [ 0.0, 0.0, 2.0,-2.0]
Observed Signal (Y) = [-0.3, 0.2, 1.8,-2.4]
Threshold (λ)       = 0.5
Noise std (σ)       = 0.5, d = 4

1. Soft Thresholding η_λ(Y) = sign(Y) * max(|Y| - λ, 0):
   y_1 = -0.3 ->  0.0
   y_2 =  0.2 ->  0.0
   y_3 =  1.8 ->  1.3
   y_4 = -2.4 -> -1.9
   η_λ(Y) = [0.0, 0.0, 1.3, -1.9]

2. Residual squared norm ||η_λ(Y) - Y||²:
   (0 - -0.3)² + (0 - 0.2)² + (1.3 - 1.8)² + (-1.9 - -2.4)²
   = 0.09 + 0.04 + 0.25 + 0.25 = 0.63

3. Divergence div η_λ(Y) (sum of indicator |y_i| > λ):
   |-0.3| <= 0.5 (0)
   |0.2|  <= 0.5 (0)
   |1.8|  >  0.5 (1)
   |-2.4| >  0.5 (1)
   div = 2

4. SURE = 0.63 + 2*(0.5²)*2 - 4*(0.5²)
        = 0.63 + 2*0.25*2 - 4*0.25
        = 0.63 + 1.0 - 1.0 = 0.63

5. Realized squared error ||η_λ(Y) - θ||²:
   (0 - 0)² + (0 - 0)² + (1.3 - 2)² + (-1.9 - -2)²
   = 0 + 0 + (-0.7)² + (0.1)²
   = 0.49 + 0.01 = 0.5

Experiment

In the Interactive Module, you can vary the underlying signal type (Sparse, Zeros, Strong), the threshold parameter λ, and the noise level σ. To reproduce the calculation:

Set the Data control to "Sparse".
Set Threshold to 0.5.
Set Noise sigma to 0.5.
Observe the computed SURE total risk estimate, which is identical to the calculated 0.63.

Observe how the SURE estimate tracks the true realized error. While the deterministic samples provided illustrate the behavior of SURE, they do not prove unbiasedness over repeated samples. Notice that SURE can sometimes be negative; it should never be artificially clamped to 0.

Limitations

Stein's Unbiased Risk Estimate is an unbiased estimator of the expected risk, meaning it is accurate on average over many realizations of the noise. For any single realization, the SURE estimate and the true realized error will differ.

References

Primary references for SURE and soft thresholding include:
Minimax Theory and its Applications (Harrison H. Zhou)

SURE Soft-Threshold Risk