Chapter 5: Sampling & Acceleration

Training a diffusion model is only half the story. Once we have a learned score function $s_\theta(x_t, t) \approx \nabla_{x_t} \log p_t(x_t)$ or equivalently a noise predictor $\epsilon_\theta(x_t, t)$, we must sample from the model---starting from pure noise $x_T \sim \mathcal{N}(0, \mathbf{I})$ and iteratively denoising until we reach a clean sample $x_0$. The original DDPM sampler requires $T = 1000$ sequential neural network evaluations, making generation painfully slow compared to GANs or VAEs that produce samples in a single forward pass.

This has spurred a rich line of research on sampling acceleration. The key insight is that the reverse process defines a probability flow ODE (ordinary differential equation) whose trajectories share the same marginal distributions as the stochastic reverse SDE. We can integrate this ODE with fewer, larger steps using sophisticated numerical methods. Beyond better ODE solvers, distillation methods train a student model to match the teacher's multi-step output in fewer steps, and consistency models learn to map any point on the ODE trajectory directly to its endpoint $x_0$.

Orthogonal to speed, classifier-free guidance is the standard technique for trading diversity for quality during sampling. By interpolating between conditional and unconditional score estimates, guidance amplifies the influence of the conditioning signal (e.g., a text prompt), producing sharper, more coherent samples at the cost of reduced diversity.

This chapter covers:

• Ancestral Sampling (DDPM): The original stochastic sampler with $T$ steps
• DDIM Deterministic Sampling: Non-Markovian generalization enabling fewer steps and deterministic generation
• DPM-Solver & Higher-Order Methods: Fast ODE solvers exploiting the semi-linear structure of diffusion ODEs
• Progressive Distillation: Training a student to halve the number of steps iteratively
• Consistency Models: Direct single-step mapping from any noise level to clean data
• Classifier-Free Guidance: Steering sample quality by blending conditional and unconditional predictions