Gaussian Diffusion Forward Process

The forward noising process in diffusion models incrementally adds Gaussian noise to an initial clean signal x_0 . By fixing a variance schedule β_t , we can compute the distribution at any timestep t in a single closed-form step without simulating intermediate steps.

Worked Example

Let the initial clean point be on the axis, x_0 = (2, 0) . We use a constant variance schedule β = 0.1 and compute the state at step t = 2 .

First, calculate α_t , which is the product of (1 - β_i) up to step t . For constant β = 0.1 and t=2 :
α₂ = (1 - 0.1) × (1 - 0.1) = 0.9 × 0.9 = 0.81 .
The conditional mean is scaled by √α₂ .
μ₂ = √0.81 × x_0 = 0.9 × (2, 0) = (1.8, 0) .
The variance at step 2 is given by 1 - α₂ = 1 - 0.81 = 0.19 .
The signal-to-noise ratio (SNR) is α₂ / (1 - α₂) = 0.81 / 0.19 ≈ 4.263 .
To generate a sample x_t , we draw noise ε ~ N(0, I) . If we draw a diagonal noise vector ε = (1, -1) , then:
x_2 = μ₂ + √0.19 × ε
x_2 = (1.8, 0) + √0.19 × (1, -1)
Since √0.19 ≈ 0.4358898943540673 , the sample is roughly (2.235889, -0.435889) .

Experiment

Observe the deterministic forward process using the module at https://riano.app/module/gaussian-diffusion-forward-process.

Set the Clean point to "Axis" (x0=2, y0=0), Beta to "0.1", Steps to "0", and Noise draw to "Diagonal". The variance is 0, giving an infinite SNR. The conditional mean exactly overlaps the sample at (2.0, 0.0).
Change Steps to "2". Notice the mean shrinks toward the origin at (1.8, 0.0), and the green covariance ellipse grows. The sample jumps diagonally from the mean by the standard deviation.
Change Steps to "6". The signal shrinks further to mean x = 1.458. The variance per coordinate grows to roughly 0.468, drastically reducing the SNR to roughly 1.134.

Limitations

This visualization only illustrates the forward noising process, not the reverse generative process (the learned denoiser). We assume a constant β variance schedule instead of common linear or cosine schedules used in practice. Furthermore, the fixed discrete noise draws (like "Diagonal") are illustrative for verifying the conditioning formulas and visualizing standard deviations, not a true Monte Carlo approximation. At t=0 , the variance evaluates to exactly zero, meaning the distribution is a point mass and the SNR is mathematically infinite.

References

The core formulation of the forward process is described in the foundational DDPM paper.

Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models.https://arxiv.org/abs/2006.11239. Introduces the DDPM formulation and describes the exact forward noising process with constant or linear β schedules.

Gaussian Diffusion Forward Process