V-Prediction Conversions

mediumPROParameterizations and the Reverse PosteriorDiffusion ModelsDiffusion Models 1

Problem Statement

Implement the v-parameterization: the velocity target used by progressive distillation, Imagen Video and Stable Diffusion 2.x, plus its inverse back to $x_0$ and $\varepsilon$ .

Background

Write $\sqrt{\bar{\alpha}_t} = \cos\phi$ and $\sqrt{1-\bar{\alpha}_t} = \sin\phi$ , so that $x_t = \cos\phi\,x_0 + \sin\phi\,\varepsilon$ traces a circular arc between the clean sample and pure noise. The velocity is the derivative of that arc:

$v = \sqrt{\bar{\alpha}_t}\,\varepsilon - \sqrt{1-\bar{\alpha}_t}\,x_0$

$v$ and $x_t$ are orthogonal unit-scaled directions, which makes the inverse a rotation by the same angle:

$\hat{x}_0 = \sqrt{\bar{\alpha}_t}\,x_t - \sqrt{1-\bar{\alpha}_t}\,v, \qquad \hat{\varepsilon} = \sqrt{1-\bar{\alpha}_t}\,x_t + \sqrt{\bar{\alpha}_t}\,v$

Note there is no division by $\sqrt{\bar{\alpha}_t}$ anywhere. That is the whole point: $\varepsilon$ -prediction degenerates at $\bar{\alpha}_t \to 0$ (the target becomes trivially $x_t$ itself, and recovering $x_0$ divides by something tiny), while $x_0$ -prediction degenerates at $\bar{\alpha}_t \to 1$ . The $v$ target is well-conditioned at both ends, which is what makes few-step distillation and zero-terminal-SNR schedules work.

Your Task

Implement two functions:

def compute_v(x0, eps, alpha_bar):
def v_to_x0_eps(x_t, v, alpha_bar):

compute_v returns the velocity target; v_to_x0_eps returns the tuple (x0, eps).

Input Format

x0, eps, x_t, v: NumPy arrays of matching shape.
alpha_bar (float): the scalar $\bar{\alpha}_t$ in (0, 1].

Output Format

compute_v returns one array; v_to_x0_eps returns a tuple of two arrays.

Sample

x0 = np.array([1.0, -1.0])
eps = np.array([0.5, 0.25])
ab = 0.36
v = compute_v(x0, eps, ab)
print(np.round(v, 4).tolist())
x_t = np.sqrt(ab) * x0 + np.sqrt(1 - ab) * eps
r0, re = v_to_x0_eps(x_t, v, ab)
print(np.round(r0, 4).tolist())
print(np.round(re, 4).tolist())

With sqrt(0.36) = 0.6 and sqrt(0.64) = 0.8, v = 0.6eps - 0.8x0, and the inverse rotation returns exactly the original x0 and eps.

Example

Input

x0 = np.array([1.0, -1.0])
eps = np.array([0.5, 0.25])
ab = 0.36
v = compute_v(x0, eps, ab)
print(np.round(v, 4).tolist())
x_t = np.sqrt(ab) * x0 + np.sqrt(1 - ab) * eps
r0, re = v_to_x0_eps(x_t, v, ab)
print(np.round(r0, 4).tolist())
print(np.round(re, 4).tolist())

Output

[-0.5, 0.95]
[1.0, -1.0]
[0.5, 0.25]

Reasoning

sqrt(0.36) = 0.6 and sqrt(0.64) = 0.8. So v = 0.6*[0.5, 0.25] - 0.8*[1.0, -1.0] = [-0.5, 0.95]. The noisy sample is x_t = 0.6*x0 + 0.8*eps = [1.0, -0.4]. Rotating back, x0_hat = 0.6*x_t - 0.8*v and eps_hat = 0.8*x_t + 0.6*v return the originals exactly, since the two maps are transposed rotations.

Constraints

0 < alpha_bar <= 1.
Both functions use the same two coefficients $\sqrt{\bar{\alpha}_t}$ and $\sqrt{1-\bar{\alpha}_t}$ -- no divisions.
Mind the signs: compute_v subtracts the $x_0$ term, and the $x_0$ recovery subtracts the $v$ term while the $\varepsilon$ recovery adds it.
The two maps must round-trip: with x_t = sqrt(ab)*x0 + sqrt(1-ab)*eps, calling v_to_x0_eps(x_t, compute_v(x0, eps, ab), ab) returns (x0, eps).
Do not round inside either function.

Learn this in

Reverse Process & DenoisingDiffusion Study Plan Sampling & AccelerationDiffusion Study Plan

More from Diffusion Models 1

Classifier-Free Guidance: Uncond/Cond Mixingeasy DDPM Posterior Mean and Variance from x0medium Adaptive CFG Scheduling over the Trajectoryhard Classifier Gradient Guidance Stepmedium Negative Prompt as the Unconditional Branchmedium

Concept primerPREVIEW

1. Background Knowledge

Diffusion models are generative models that learn to reverse a gradual noising process. In the standard formulation, a clean data point $x_0$ is corrupted by Gaussian noise $\varepsilon$ over $T$ steps. The state at any intermediate step $t$ is defined as $x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1-\bar{\alpha}_t} \varepsilon$ . Here, $\bar{\alpha}_t$ represents the cumulative product of noise schedule parameters, effectively controlling how much of the original signal remains versus how much noise has been added.

Traditionally, models predict either the noise $\varepsilon$ or the clean data $x_0$ . However, these parameterizations suffer from numerical instability at the extremes of the diffusion process. Predicting $\varepsilon$ becomes ill-conditioned when $\bar{\alpha}_t \to 0$ (high noise), while predicting $x_0$ becomes ill-conditioned when $\bar{\alpha}_t \to 1$ (low noise). The v-parameterization addresses this by predicting the "velocity" vector $v$ , which is tangent to the circular arc connecting $x_0$ and $\varepsilon$ . This approach ensures stable gradients and accurate predictions throughout the entire diffusion trajectory, making it ideal for advanced techniques like progressive distillation.

The mathematical foundation relies on a rotation interpretation. If we define $\cos\phi = \sqrt{\bar{\alpha}_t}$ and $\sin\phi = \sqrt{1-\bar{\alpha}_t}$ , the forward process is a rotation of the vector $(x_0, \varepsilon)$ . The velocity $v$ is defined as $v = \sqrt{\bar{\alpha}_t}\varepsilon - \sqrt{1-\bar{\alpha}_t}x_0$ . Crucially, recovering $x_0$ and $\varepsilon$ from $x_t$ and $v$ involves a simple inverse rotation, requiring no division by small numbers. This geometric symmetry is what provides the numerical stability.

2. Algorithm Approach

The core algorithmic task is to implement two linear transformations based on the provided trigonometric identities. Since the operations are element-wise and involve only basic arithmetic (multiplication, addition, subtraction) and square roots, the approach is straightforward vectorization.

For compute_v, you must calculate the weighted difference between the noise and the clean data. The weights are determined by the square root of the noise schedule parameter $\bar{\alpha}_t$ and its complement $1-\bar{\alpha}_t$ .

For v_to_x0_eps, you must perform the inverse transformation. This is essentially a rotation matrix multiplication. Given the noisy sample $x_t$ and the predicted velocity $v$ , you combine them using the same weights but with different signs to isolate $x_0$ and $\varepsilon$ . The key insight is that this is a linear system that can be solved directly without iterative methods or matrix inversions.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.