Variational KL and Exponential Tilting

Worked Example

Consider the case of "Crossed" laws over a three-point domain, with a "Linear" score, Scale of 1, and Offset of 0. We have the probability distributions: P = (0.5, 0.3, 0.2) and Q = (0.2, 0.5, 0.3). The score function f is chosen as [0, 1, 2].

Let's trace the calculation step by step:

1. Calculate Expected Score under P:
   E_P[f] = 0.5 * 0 + 0.3 * 1 + 0.2 * 2 = 0.7

2. Calculate Partition Function Z under Q:
   Z = E_Q[exp(f)] = 0.2*exp(0) + 0.5*exp(1) + 0.3*exp(2)
     ≈ 0.2*1 + 0.5*2.7183 + 0.3*7.3891 ≈ 3.7758...
   log Z = log(3.7758) ≈ 1.3286275738205926

3. Calculate Variational Bound B:
   B = E_P[f] - log Z = 0.7 - 1.3286275738205926
     = -0.6286275738205926

4. Calculate True KL(P||Q):
   KL = 0.5*log(0.5/0.2) + 0.3*log(0.3/0.5) + 0.2*log(0.2/0.3)
      ≈ 0.4581 - 0.1532 - 0.0811 ≈ 0.22380465718564743

5. Calculate Tilted First Probability (R_0):
   R_0 = Q_0 * exp(f_0) / Z = 0.2 * 1 / 3.7758 ≈ 0.05296809720192549

The Bound gap is therefore 0.2238 - (-0.6286) = 0.8524. Gibbs' inequality guarantees that the bound B is always less than or equal to the True KL divergence.

Experiment

You can reproduce this behavior using the interactive module controls. Open the module at https://riano.app/module/variational-kl-exponential-tilt and follow these steps:

1. Set the Laws control to "Crossed".
2. Set the Score control to "Log density ratio".
3. Observe that the Bound gap drops to exactly 0.0. The variational bound is tight (equal to the True KL) when the chosen score is precisely the log density ratio log(P/Q).
4. Now change the Scale control to "2" and note that the bound becomes looser (Bound gap > 0).
5. Adjust the Offset control (e.g., to "1" or "-1"). Notice that the Log partition changes, but the Variational bound, True KL, Bound gap, and the tilted probabilities (R) remain completely unaffected by additive constants.

Limitations

This module considers only positive three-point discrete distributions to ensure analytical exactness and visualize all states without approximation. In high-dimensional spaces or continuous domains, computing the exact partition function log Z is often computationally intractable, requiring Monte Carlo approximation or specialized architectures.

Numerically, the implementation uses natural logarithms and standard floating-point precision. Analytically zero gaps can appear as tiny values bounded by roughly 1e-14 due to floating point roundoff. We clamp differences below 1e-11 to true zero in the provided examples to reflect the algebraic reality of equal laws or optimal scores. Equal laws do not make every nonconstant score tight; if P=Q but the score is varying, the resulting tilted law R diverges from Q, opening a bound gap.

References

The concept of bounding the Kullback-Leibler divergence using arbitrary scores forms the basis of many variational approximation techniques. The mathematical formulation, particularly the relationship between the partition function and the exponential family tilting, relies heavily on convex analysis and properties of the log-sum-exp function. By providing a tractable alternative to calculating integrals over intractable distributions, these bounds are central to methods such as variational autoencoders (VAEs) and modern approximate Bayesian inference. Below is a primary textbook reference outlining these exact foundations:

• Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Explains conjugate functions, log-sum-exp, and the mathematical foundations behind variational lower bounds and exponential families.