Entropic Transport Coupling Bias

Entropic optimal transport modifies the classical optimal transport problem by adding a Kullback-Leibler divergence penalty against a reference independence coupling. This regularization makes the problem strictly convex and efficiently solvable via matrix scaling (Sinkhorn's algorithm), but it introduces a bias that shifts the optimal transport plan away from the unregularized minimum-cost coupling toward the independence coupling.

Worked Example: Diagonal Free Cost

Consider two marginal distributions which are both uniform on two points, yielding independent uniform probabilities of 1/4. We define a "Diagonal free" cost matrix with gap d=1:
C = [[0, 1], [1, 0]]

With regularization parameter ε = 0.5, we want to find the transport plan P minimizing ⟨ C, P ⟩ + ε KL(P || uniform). Let's trace the calculation:

Compute the scaling factor z = exp(-d / ε) = exp(-1 / 0.5) = exp(-2) ≈ 0.135335.
Because of symmetry, the diagonal entries of P are a = 0.5 / (1 + z) ≈ 0.5 / 1.135335 ≈ 0.4403985.
The off-diagonal entries are b = 0.5 · z / (1 + z) ≈ 0.5 · 0.135335 / 1.135335 ≈ 0.0596014.
The optimal plan P is thus: [[0.4403985, 0.0596014], [0.0596014, 0.4403985]].
The total transport cost is 0 · 0.4403985 · 2 + 1 · 0.0596014 · 2 ≈ 0.1192029.
The unregularized minimum cost would be 0 (with P = [[0.5, 0], [0, 0.5]]), meaning the regularization has introduced a transport cost bias of 0.1192029.

Experimenting with Regularization

You can interact with the Entropic Transport Coupling Bias module (https://riano.app/module/entropic-transport-coupling-bias) to observe how different settings affect the coupling:

Set Cost matrix to "Diagonal free" and Cost gap to 1.
Change Regularization epsilon from 0 to 2. Notice how the transport cost increases from 0 (at epsilon 0) as the coupling is drawn toward the uniform independent distribution.
Change Cost matrix to "Cross free". The plan matrix swaps its diagonal and off-diagonal entries, matching the new cost structure while preserving the exact same regularized objective value.
Set Cost matrix to "Flat". Observe that the transport cost equals the cost gap regardless of epsilon, and at epsilon=0 the minimizer becomes nonunique.

Limitations and Assumptions

The entropic regularization introduced here assumes a fixed independent reference coupling. While it makes the optimal transport problem efficiently solvable, the resulting regularized objective is no longer a true metric (it does not satisfy the triangle inequality and can be non-zero for comparing a distribution with itself). To recover an unbiased distance (such as the Sinkhorn divergence), additional debiasing terms subtracting the self-entropic transport costs are required. Furthermore, very small values of ε can lead to numerical underflow in practice if the exponential scaling is not handled in the log domain.

References

For a comprehensive mathematical foundation of entropic regularization in optimal transport and the impact of the bias, see Cuturi (2013), "Sinkhorn Distances: Lightspeed Computation of Optimal Transport":https://arxiv.org/abs/1306.0895.

Entropic Transport Coupling Bias