Classifier-Free Guidance Combination
Problem Statement
Combine a conditional and an unconditional noise prediction into the guided prediction that every text-to-image sampler actually uses.
Background
Classifier-free guidance (Ho & Salimans, 2022) drops the conditioning at random during training so one network learns both ε^θ​(xt​,t,c) and ε^θ​(xt​,t,∅). At sampling time the two are extrapolated:
ε^guided​=ε^θ​(xt​,t,∅)+w(ε^θ​(xt​,t,c)−ε^θ​(xt​,t,∅))
The bracket is (proportional to) the score of the implicit classifier p(c∣xt​): the direction in noise space that makes the sample more like the prompt. Guidance walks along it.
Read off the boundary cases, because they are the whole intuition: w=0 gives the unconditional model, w=1 gives the plain conditional model (no guidance at all), and w>1 extrapolates past the conditional prediction. The usual Stable Diffusion default of guidance_scale = 7.5 is therefore a 7.5x overshoot, trading diversity for prompt fidelity. Note that w = 1 meaning "off" is exactly why some codebases skip the second forward pass when guidance_scale <= 1.
Your Task
Implement:
def cfg_combine(eps_uncond, eps_cond, w):
Return the guided noise prediction, shaped like the inputs.
Input Format
- eps_uncond, eps_cond: NumPy arrays of matching shape.
- w (float): the guidance scale.
Output Format
A NumPy array shaped like the inputs.
Sample
u = np.array([0.1, -0.2])
c = np.array([0.3, 0.2])
print(np.round(cfg_combine(u, c, 7.5), 4).tolist())
The difference is [0.2, 0.4], so the result is [0.1 + 7.50.2, -0.2 + 7.50.4] = [1.6, 2.8].
Example:
u = np.array([0.1, -0.2]) c = np.array([0.3, 0.2]) print(np.round(cfg_combine(u, c, 7.5), 4).tolist())
[1.6, 2.8]
The conditional direction is c - u = [0.2, 0.4]. Guidance walks 7.5 times along it from the unconditional base: 0.1 + 7.5*0.2 = 1.6 and -0.2 + 7.5*0.4 = 2.8. Both components are pushed well past the conditional prediction itself -- that overshoot is exactly what a guidance scale above 1 means.
Constraints:
- The base of the extrapolation is the unconditional prediction; the scale multiplies the difference.
wmay be any float, including0.0and values below 1.w = 1.0must returneps_condexactly.- Do not round inside the function.
1. Background Knowledge
Classifier-Free Guidance (CFG) is a technique used in diffusion models to improve sample quality without requiring a separate classifier. During training, the model learns to predict noise conditioned on a prompt c and also unconditionally (when the prompt is dropped). At inference time, we leverage both predictions to steer the generation process toward the desired condition.
The core intuition is that the difference between the conditional prediction ε^θ​(xt​,t,c) and the unconditional prediction ε^θ​(xt​,t,∅) represents the direction in latent space that makes the sample more aligned with the prompt. By scaling this difference by a guidance weight w, we can control how strongly the model adheres to the prompt.
The formula for combining these predictions is:
ε^guided​=ε^θ​(xt​,t,∅)+w(ε^θ​(xt​,t,c)−ε^θ​(xt​,t,∅))This can be rewritten as:
ε^guided​=(1+w)ε^θ​(xt​,t,c)−wε^θ​(xt​,t,∅)When w=0, the result is purely unconditional. When w=1, it is purely conditional. For w>1, the model extrapolates beyond the conditional prediction, increasing fidelity to the prompt at the cost of diversity.
2. Algorithm Approach
The problem requires implementing a simple vector arithmetic operation based on the CFG formula. The approach involves:
- Input Validation: Ensure inputs are NumPy arrays of the same shape.
- Vector Operations: Use NumPy's broadcasting and element-wise operations to compute the guided prediction.
- Return Result: Return the resulting array.
The key is to leverage NumPy's efficient array operations to avoid explicit loops, ensuring the solution is both concise and performant.
3. Step-by-Step Strategy
- Import NumPy: Ensure numpy is imported as np.
- Define Function: Create the function cfg_combine(eps_uncond, eps_cond, w).
- Compute Difference: Calculate the difference between the conditional and unconditional predictions: diff = eps_cond - eps_uncond.
- Apply Guidance: Multiply the difference by the guidance scale w: guided_diff = w * diff.
- Combine Predictions: Add the scaled difference to the unconditional prediction: result = eps_uncond + guided_diff.
- Return Result: Return the result array.
Alternatively, you can use the rewritten form:
result = (1 + w) * eps_cond - w * eps_uncond
This is mathematically equivalent and may be slightly more efficient.
4. Common Pitfalls
- Shape Mismatch: Ensure eps_uncond and eps_cond have the same shape. NumPy will raise an error if they do not.
- Data Types: Be mindful of data types. If inputs are integers, the result may be truncated. Use floating-point arrays for precision.
- Guidance Scale: Understand the impact of w. Values much greater than 1 can lead to over-saturation or artifacts in generated images.
- In-place Operations: Avoid modifying input arrays in-place unless explicitly required, as this can lead to unexpected side effects.
5. Time & Space Complexity
- Time Complexity: O(N), where N is the number of elements in the input arrays. Each element is processed once during the arithmetic operations.
- Space Complexity: O(N), for storing the result array. If using the rewritten form, intermediate arrays may be created, but NumPy optimizes this efficiently.