Chapter 7: Conditional Generation

Unconditional diffusion models generate impressive samples, but most practical applications require control over what is generated. Conditional generation bridges this gap by steering the diffusion process toward outputs that match a given specification---a text prompt, a class label, a reference image, or a masked region to fill. This chapter explores the full spectrum of conditioning techniques that transformed diffusion models from curiosity to creative powerhouse.

The journey begins with classifier guidance, which demonstrated that gradient signals from an external classifier could dramatically improve sample quality and class adherence. Classifier-free guidance then eliminated the need for a separate classifier entirely, becoming the dominant conditioning mechanism in modern systems. These guidance techniques underpin every major text-to-image model: DALL-E 2, Imagen, and Stable Diffusion each combine guidance with different text encoders and architectural choices to translate natural language into pixels.

Beyond text-to-image, conditional diffusion enables a rich family of image manipulation tasks. Image-to-image translation lets you transform existing images while preserving structure. Inpainting and outpainting fill or extend images seamlessly using masked diffusion. Super-resolution cascades push outputs to megapixel resolutions through iterative upsampling. Together, these techniques form the foundation of modern AI-powered image editing.

This chapter covers:

• Classifier Guidance: Steering generation with external classifier gradients
• Classifier-Free Guidance: Training a single model for both conditional and unconditional generation
• Text-to-Image Generation: Architectures behind DALL-E 2, Imagen, and Stable Diffusion
• Image-to-Image Translation: SDEdit and controlled image transformation
• Inpainting & Outpainting: Masked diffusion for filling and extending images
• Super-Resolution: Cascaded diffusion for high-resolution image synthesis