Machine Learning

Classifier-Free Gaussian Guidance

Check when an extrapolated score is the gradient of a normalizable density.

By Riano · Published · Updated · Report a correction

Classifier-Free Gaussian Guidance

Classifier-free guidance extrapolates between an unconditional score and a conditional score to improve sample quality. We analyze when this extrapolated score remains the gradient of a valid, normalizable probability density.

Worked Example

Consider an unconditional density N(0, 1) and a conditional density N(1, 4) (where sigma = 2). The guidance weight is w = 1. We compute the scores at x = 0.

  1. Unconditional score s_uncond(x) = -x. At x = 0, s_uncond(0) = 0.
  2. Conditional score s_cond(x) = (1 - x)/sigma2. At x = 0 with sigma = 2, s_cond(0) = 1/4 = 0.25.
  3. Guided score s_guided(x) = s_uncond(x) + w * (s_cond(x) - s_uncond(x)).
  4. s_guided(0) = 0 + 1 * (0.25 - 0) = 0.25.
  5. Precision A = 1 - w + w/sigma2 = 1 - 1 + 1/4 = 0.25.
  6. Mean = B/A where B = w/sigma2 = 1/4 = 0.25. Mean = 0.25 / 0.25 = 1.0.
  7. Variance = 1/A = 1/0.25 = 4.0.
  8. Since A > 0, the density is normalizable. The density of N(1.0, 4.0) at x = 0 is 1/sqrt(8*pi) * exp(-1/8) = 0.17603266338214976.

Experiment

Open the Classifier-Free Gaussian Guidance module to observe how guidance weight and variance affect the resulting distribution.

  1. Set Conditional sigma to "2" and Guidance weight to "1". The Density status is Normalizable.
  2. Increase Guidance weight to "4/3". The Guided precision drops to 0.0, and the status changes to Not normalizable.
  3. Change Conditional sigma to "0.5" and Guidance weight to "2". The precision becomes 7.0 (a very sharp density with variance ~0.14).
  4. Change Probe x to verify the scores at different points while the global density parameters remain the same.

Assumptions and Limitations

This analysis assumes fixed, static Gaussian slices and does not prove that a full diffusion sampling trajectory perfectly tracks this exact distribution.

The formal density is proportional to p_uncond1-w * p_condw. This normalizes to a Gaussian ONLY if the precision A > 0. When A = 0 or A is negative (e.g. sigma = 2, w = 4/3 gives A = 0), there is no normalized density, meaning the moments and density are Unavailable, although the score function itself remains finite. Strong extrapolation can therefore be non-normalizable instead of just sharper.

References

The foundation of classifier-free guidance and score extrapolation.

  • Ho, J., & Salimans, T. (2022). Classifier-Free Diffusion Guidance. This paper introduces the formulation w = 1 + guidanceScale and discusses generating samples without a classifier.

Experience it interactively

Adjust parameters, observe in real time, and build deep intuition with Riano’s interactive Classifier-Free Gaussian Guidance module.

Try Classifier-Free Gaussian Guidance on Riano →

More in Machine Learning

Gradient Descent

Finding the minimum of a cost function.

Perceptron

Visualize the Perceptron algorithm learning a linear decision boundary.

K-Means Clustering

Partitioning data into clusters by iteratively assigning points and updating centroids.