Loading...
Fix the over-exposure artefact of high guidance scales by rescaling the guided prediction back to the standard deviation of the conditional one, then blending.
Extrapolating with w≫1 inflates the magnitude of the noise prediction: std(ε^cfg) grows roughly linearly in w. Since the sampler treats that output as a unit-variance noise estimate, the extra magnitude is spent pushing the sample toward the extremes of the data range -- the blown-out, over-saturated look of high-CFG images.
Common Diffusion Noise Schedules and Sample Steps are Flawed (Lin et al., 2024) fixes this in two moves. First renormalise:
\hat{\varepsilon}_{\text{rescaled}} = \hat{\varepsilon}_{\text{cfg}} \cdot \frac{\mathrm{std}(\hat{\varepsilon}_c)}{\mathrm{std}(\hat{\varepsilon}_{\text{cfg}})}$$ Full renormalisation turns out to over-correct and flatten the image, so the paper interpolates back with a factor $\phi$ (the **guidance_rescale** argument, typically 0.7): $$\hat{\varepsilon}_{\text{final}} = \phi\,\hat{\varepsilon}_{\text{rescaled}} + (1-\phi)\,\hat{\varepsilon}_{\text{cfg}}$$ The standard deviations are computed **per sample**, over all non-batch dimensions, as the population std (**ddof=0**, NumPy's default). ## Your Task Implement: ```python def cfg_rescale(eps_uncond, eps_cond, w, phi=0.7): ``` Inputs are 2-D arrays of shape **(N, D)**; return an array of the same shape. ## Input Format - **eps_uncond**, **eps_cond**: arrays of shape **(N, D)**. - **w** (float): guidance scale. - **phi** (float): rescale blend factor in **[0, 1]**. ## Output Format An array of shape **(N, D)**. ## Sample ```python u = np.array([[0.0, 1.0, -1.0, 0.0]]) c = np.array([[0.0, 2.0, -2.0, 0.0]]) print(np.round(cfg_rescale(u, c, 3.0, 0.7), 4).tolist()) ``` The guided prediction is **[0, 4, -4, 0]** with std 2.8284, while the conditional has std 1.4142, so the rescaled version is halved to **[0, 2, -2, 0]**, and the 0.7 blend gives **[0, 2.6, -2.6, 0]**.u = np.array([[0.0, 1.0, -1.0, 0.0]]) c = np.array([[0.0, 2.0, -2.0, 0.0]]) print(np.round(cfg_rescale(u, c, 3.0, 0.7), 4).tolist())
[[0.0, 2.6, -2.6, 0.0]]
eps_cfg = u + 3*(c - u) = [0, 4, -4, 0], whose population std is 2.8284, while std(c) = 1.4142. The ratio 0.5 halves it to [0, 2, -2, 0]. Blending with phi = 0.7 gives 0.7*[0,2,-2,0] + 0.3*[0,4,-4,0] = [0, 2.6, -2.6, 0] -- most of the magnitude inflation removed, but not all.
ddof=0), NumPy's default.phi = 0.0 must reproduce plain classifier-free guidance exactly.std values are guaranteed non-zero for the given inputs.