Envelope Theorem

Differentiate an optimized value while the optimizer moves and constraints become active.

Envelope Theorem: Derivatives of Optimized Values

When an objective function depends on a parameter, finding how its optimized value changes with respect to that parameter might seem to require differentiating the entire optimization process. The envelope theorem shows that, under certain conditions, we only need to evaluate the partial derivative of the objective function at the optimal point.

Worked example: Differentiating an Optimal Value

Consider the objective function parameterized by a reward a and a curvature b: V(a) = max_{x ∈ [0, C]} (a·x - 0.5·b·x²). Let a = 1, b = 1, and C = 2.

First, find the unconstrained maximum by taking the derivative with respect to x: d/dx (a·x - 0.5·b·x²) = a - b·x = 0, which gives x = a/b = 1/1 = 1.
Check the constraints x ∈ [0, 2]. The unconstrained optimum x = 1 is strictly inside the bounds, so the optimal action x^* is 1.
Calculate the optimal value V(1) = 1·(1) - 0.5·(1)·(1)² = 0.5.
To find the derivative of V(a) with respect to a, we apply the envelope theorem: dV/da = ∂/∂a (a·x - 0.5·b·x²) evaluated at x = x^*. This gives x^*, so dV/da = 1.

Observe the Envelope Derivative

Open Envelope Theorem to see the interaction between the parameter, the optimizer, and the optimal value.

Set Reward a to 1, Curvature b to 1, and Capacity to 2. Check that the Optimal action is 1 and the Envelope derivative is 1.
Change Reward a to 4. The unconstrained optimum would be 4, but due to the capacity constraint, the Optimal action clips to 2. Notice that the Envelope derivative becomes 2, matching the new clipped x^*.
Set Reward a to -1. The optimal action clips to the lower bound 0, and the Envelope derivative becomes 0.

Limitations

The simple statement of the envelope theorem holds for unconstrained optimization or when constraints are inactive (interior solutions). When constraints are active and binding, the optimizer's dependence on the parameter can introduce regime changes. At these boundaries, the first derivative of the value function usually remains continuous, but the second derivative can jump. This module uses a piecewise linear or quadratic function, meaning numerical derivatives must handle non-differentiable points carefully.

References

See Convex Optimization by Stephen Boyd and Lieven Vandenberghefor a comprehensive treatment of sensitivity analysis and how optimal values respond to parameter perturbations.