Classifier-Free Guidance Combination

easyFREEGuidance and Latent DiffusionDiffusion ModelsDiffusion Models 1

Problem Statement

Combine a conditional and an unconditional noise prediction into the guided prediction that every text-to-image sampler actually uses.

Background

Classifier-free guidance (Ho & Salimans, 2022) drops the conditioning at random during training so one network learns both $\hat{\varepsilon}_\theta(x_t, t, c)$ and $\hat{\varepsilon}_\theta(x_t, t, \varnothing)$ . At sampling time the two are extrapolated:

$\hat{\varepsilon}_{\text{guided}} = \hat{\varepsilon}_\theta(x_t, t, \varnothing) + w\left(\hat{\varepsilon}_\theta(x_t, t, c) - \hat{\varepsilon}_\theta(x_t, t, \varnothing)\right)$

The bracket is (proportional to) the score of the implicit classifier $p(c \mid x_t)$ : the direction in noise space that makes the sample more like the prompt. Guidance walks along it.

Read off the boundary cases, because they are the whole intuition: $w = 0$ gives the unconditional model, $w = 1$ gives the plain conditional model (no guidance at all), and $w > 1$ extrapolates past the conditional prediction. The usual Stable Diffusion default of guidance_scale = 7.5 is therefore a 7.5x overshoot, trading diversity for prompt fidelity. Note that w = 1 meaning "off" is exactly why some codebases skip the second forward pass when guidance_scale <= 1.

Your Task

Implement:

def cfg_combine(eps_uncond, eps_cond, w):

Return the guided noise prediction, shaped like the inputs.

Input Format

eps_uncond, eps_cond: NumPy arrays of matching shape.
w (float): the guidance scale.

Output Format

A NumPy array shaped like the inputs.

Sample

u = np.array([0.1, -0.2])
c = np.array([0.3, 0.2])
print(np.round(cfg_combine(u, c, 7.5), 4).tolist())

The difference is [0.2, 0.4], so the result is [0.1 + 7.50.2, -0.2 + 7.50.4] = [1.6, 2.8].

Example

Input

u = np.array([0.1, -0.2])
c = np.array([0.3, 0.2])
print(np.round(cfg_combine(u, c, 7.5), 4).tolist())

Output

[1.6, 2.8]

Reasoning

The conditional direction is c - u = [0.2, 0.4]. Guidance walks 7.5 times along it from the unconditional base: 0.1 + 7.5*0.2 = 1.6 and -0.2 + 7.5*0.4 = 2.8. Both components are pushed well past the conditional prediction itself -- that overshoot is exactly what a guidance scale above 1 means.

Constraints

The base of the extrapolation is the unconditional prediction; the scale multiplies the difference.
w may be any float, including 0.0 and values below 1.
w = 1.0 must return eps_cond exactly.
Do not round inside the function.

Learn this in

Sampling & AccelerationDiffusion Study Plan Conditional GenerationDiffusion Study Plan Multimodal LLMsLLM Study Plan

More from Diffusion Models 1

Classifier-Free Guidance: Uncond/Cond Mixingeasy DDPM Posterior Mean and Variance from x0medium Adaptive CFG Scheduling over the Trajectoryhard Classifier Gradient Guidance Stepmedium Negative Prompt as the Unconditional Branchmedium

Concept primer

1. Background Knowledge

Classifier-Free Guidance (CFG) is a technique used in diffusion models to improve sample quality without requiring a separate classifier. During training, the model learns to predict noise conditioned on a prompt $c$ and also unconditionally (when the prompt is dropped). At inference time, we leverage both predictions to steer the generation process toward the desired condition.

The core intuition is that the difference between the conditional prediction $\hat{\varepsilon}_\theta(x_t, t, c)$ and the unconditional prediction $\hat{\varepsilon}_\theta(x_t, t, \varnothing)$ represents the direction in latent space that makes the sample more aligned with the prompt. By scaling this difference by a guidance weight $w$ , we can control how strongly the model adheres to the prompt.

The formula for combining these predictions is:

\hat{\varepsilon}_{\text{guided}} = \hat{\varepsilon}_\theta(x_t, t, \varnothing) + w\left(\hat{\varepsilon}_\theta(x_t, t, c) - \hat{\varepsilon}_\theta(x_t, t, \varnothing)\right)

This can be rewritten as:

\hat{\varepsilon}_{\text{guided}} = (1 + w)\hat{\varepsilon}_\theta(x_t, t, c) - w\hat{\varepsilon}_\theta(x_t, t, \varnothing)

When $w = 0$ , the result is purely unconditional. When $w = 1$ , it is purely conditional. For $w > 1$ , the model extrapolates beyond the conditional prediction, increasing fidelity to the prompt at the cost of diversity.

2. Algorithm Approach

The problem requires implementing a simple vector arithmetic operation based on the CFG formula. The approach involves:

Input Validation: Ensure inputs are NumPy arrays of the same shape.
Vector Operations: Use NumPy's broadcasting and element-wise operations to compute the guided prediction.
Return Result: Return the resulting array.

The key is to leverage NumPy's efficient array operations to avoid explicit loops, ensuring the solution is both concise and performant.

3. Step-by-Step Strategy

Import NumPy: Ensure numpy is imported as np.
Define Function: Create the function cfg_combine(eps_uncond, eps_cond, w).
Compute Difference: Calculate the difference between the conditional and unconditional predictions: diff = eps_cond - eps_uncond.
Apply Guidance: Multiply the difference by the guidance scale $w$ : guided_diff = w * diff.
Combine Predictions: Add the scaled difference to the unconditional prediction: result = eps_uncond + guided_diff.
Return Result: Return the result array.

Alternatively, you can use the rewritten form:

result = (1 + w) * eps_cond - w * eps_uncond

This is mathematically equivalent and may be slightly more efficient.

4. Common Pitfalls

Shape Mismatch: Ensure eps_uncond and eps_cond have the same shape. NumPy will raise an error if they do not.
Data Types: Be mindful of data types. If inputs are integers, the result may be truncated. Use floating-point arrays for precision.
Guidance Scale: Understand the impact of $w$ . Values much greater than 1 can lead to over-saturation or artifacts in generated images.
In-place Operations: Avoid modifying input arrays in-place unless explicitly required, as this can lead to unexpected side effects.

5. Time & Space Complexity

Time Complexity: $O(N)$ , where $N$ is the number of elements in the input arrays. Each element is processed once during the arithmetic operations.
Space Complexity: $O(N)$ , for storing the result array. If using the rewritten form, intermediate arrays may be created, but NumPy optimizes this efficiently.

solution.py

Python 3.13

Test results