Linear Beta Schedule and Alpha-Bar

easyFREEForward Process and Noise SchedulesDiffusion ModelsDiffusion Models 1

Problem Statement

Build the linear noise schedule used by the original DDPM paper and derive the two quantities every later formula depends on: the per-step signal retention alpha_t and the cumulative signal retention alpha_bar_t.

Background

The forward diffusion process corrupts data one small step at a time:

$q(x_t \mid x_{t-1}) = \mathcal{N}\!\left(x_t;\ \sqrt{1-\beta_t}\,x_{t-1},\ \beta_t I\right)$

The variances $\beta_1, \dots, \beta_T$ are the noise schedule. DDPM uses a linear schedule: $T$ values spaced evenly (endpoints included) from beta_start to beta_end, with the defaults $\beta_1 = 10^{-4}$ and $\beta_T = 0.02$ for $T = 1000$ .

From the betas we define

$\alpha_t = 1 - \beta_t, \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s$

$\bar{\alpha}_t$ is the running product, not a running sum. It is what makes the forward process collapse into a single closed-form Gaussian, so getting it right is the foundation of everything else in diffusion.

Your Task

Implement:

def linear_schedule(T, beta_start=1e-4, beta_end=0.02):

Return a tuple (betas, alphas, alpha_bars) of three 1-D NumPy arrays, each of length T, using 0-based indexing (element i corresponds to timestep $t = i+1$ in the maths above).

Input Format

T (int): number of diffusion steps, T >= 1.
beta_start, beta_end (float): first and last beta. Both endpoints must appear in the array.

Output Format

A tuple of three NumPy arrays (betas, alphas, alpha_bars).

Sample

b, a, ab = linear_schedule(5, 0.1, 0.5)
print(np.round(b, 4).tolist())
print(np.round(a, 4).tolist())
print(np.round(ab, 4).tolist())

[0.1, 0.2, 0.3, 0.4, 0.5]
[0.9, 0.8, 0.7, 0.6, 0.5]
[0.9, 0.72, 0.504, 0.3024, 0.1512]

Betas are evenly spaced from 0.1 to 0.5, alphas are 1 - beta, and alpha_bars is their cumulative product, so it decays quickly toward zero.

Example

Input

b, a, ab = linear_schedule(5, 0.1, 0.5)
print(np.round(b, 4).tolist())
print(np.round(a, 4).tolist())
print(np.round(ab, 4).tolist())

Output

[0.1, 0.2, 0.3, 0.4, 0.5]
[0.9, 0.8, 0.7, 0.6, 0.5]
[0.9, 0.72, 0.504, 0.3024, 0.1512]

Reasoning

np.linspace(0.1, 0.5, 5) gives betas [0.1, 0.2, 0.3, 0.4, 0.5], so alphas are [0.9, 0.8, 0.7, 0.6, 0.5]. The cumulative product runs 0.9, 0.9*0.8 = 0.72, 0.72*0.7 = 0.504, 0.504*0.6 = 0.3024, 0.3024*0.5 = 0.1512 -- the signal that survives to each step. Using a cumulative sum instead would give values above 1, which is impossible for a variance-preserving process.

Constraints

T >= 1; when T == 1 the array holds beta_start only.
betas must include both endpoints (np.linspace, not np.arange).
alpha_bars is a cumulative product, not a cumulative sum.
Use float (float64) arrays; do not round inside the function.

Learn this in

Introduction to Generative ModelsDiffusion Study Plan Forward Diffusion ProcessDiffusion Study Plan

More from Diffusion Models 1

Classifier-Free Guidance: Uncond/Cond Mixingeasy DDPM Posterior Mean and Variance from x0medium Adaptive CFG Scheduling over the Trajectoryhard Classifier Gradient Guidance Stepmedium Negative Prompt as the Unconditional Branchmedium

Concept primer

1. Background Knowledge

Diffusion models are generative models that learn to reverse a gradual noising process. The forward process adds Gaussian noise to data $x_0$ over $T$ steps until it becomes pure noise $x_T$ . This process is defined by a noise schedule, typically a sequence of variances $\beta_1, \dots, \beta_T$ . In the original DDPM paper, a linear schedule is used, where $\beta_t$ increases linearly from a small value $\beta_{\text{start}}$ to a larger value $\beta_{\text{end}}$ .

The quantity $\alpha_t = 1 - \beta_t$ represents the signal retention at step $t$ . It indicates how much of the previous signal $x_{t-1}$ is preserved in $x_t$ . The cumulative signal retention $\bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s$ is crucial because it allows sampling $x_t$ directly from $x_0$ using the reparameterization trick: $x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon$ . This closed-form expression relies entirely on the correct computation of $\bar{\alpha}_t$ .

In implementation, we use 0-based indexing for arrays, so index $i$ corresponds to timestep $t = i+1$ . The array betas contains $\beta_1, \dots, \beta_T$ , alphas contains $\alpha_1, \dots, \alpha_T$ , and alpha_bars contains $\bar{\alpha}_1, \dots, \bar{\alpha}_T$ . Understanding the difference between element-wise operations and cumulative operations is key.

2. Algorithm Approach

The problem requires generating three related arrays based on a linear interpolation and cumulative product. The approach involves three main stages:

Generate Betas: Create an array of $T$ values linearly spaced between beta_start and beta_end. This can be done using NumPy's linspace function.
Compute Alphas: Apply the element-wise transformation $\alpha_t = 1 - \beta_t$ to the betas array.
Compute Alpha Bars: Calculate the cumulative product of the alphas array. This means each element at index $i$ is the product of all elements from index $0$ to $i$ .

This approach leverages vectorized operations in NumPy for efficiency and clarity.

3. Step-by-Step Strategy

Import NumPy: Ensure numpy is imported as np.
Create betas: Use np.linspace(beta_start, beta_end, T) to generate the linear schedule. This function returns an array of $T$ evenly spaced values, including both endpoints.
Create alphas: Subtract the betas array from 1 using vectorized subtraction: alphas = 1 - betas.
Create alpha_bars: Use np.cumprod(alphas) to compute the cumulative product. This function returns an array where the $i$ -th element is the product of all elements up to and including index $i$ .
Return Tuple: Return the tuple (betas, alphas, alpha_bars).

Example code structure:

import numpy as np

def linear_schedule(T, beta_start=1e-4, beta_end=0.02):
 betas = np.linspace(beta_start, beta_end, T)
 alphas = 1 - betas
 alpha_bars = np.cumprod(alphas)
 return betas, alphas, alpha_bars

4. Common Pitfalls

Indexing Confusion: The problem uses 0-based indexing for arrays but 1-based indexing for mathematical timesteps. Ensure that betas corresponds to $\beta_1$ , betas to $\beta_2$ , etc. The linspace function naturally aligns with this if you request $T$ points.
Cumulative Sum vs. Product: Do not use np.cumsum for alpha_bars. The definition is a product $\prod_{s=1}^{t} \alpha_s$ , so np.cumprod is required. Using sum will yield incorrect results.
Endpoint Inclusion: Ensure that both beta_start and beta_end are included in the betas array. np.linspace includes endpoints by default, but be cautious if using other methods like np.arange or manual loops.
Data Types: Ensure the arrays are of floating-point type. While np.linspace returns floats by default, explicit casting might be needed in some contexts, though usually not for this problem.
Off-by-One Errors: When implementing manually without cumprod, ensure the loop starts correctly and accumulates the product properly. Using cumprod avoids this issue.

5. Time & Space Complexity

Time Complexity: $O(T)$ . Generating betas with linspace takes $O(T)$ . Computing alphas is an element-wise operation taking $O(T)$ . Computing alpha_bars with cumprod is also $O(T)$ . Thus, the total time complexity is linear in $T$ .
Space Complexity: $O(T)$ . We store three arrays of length $T$ : betas, alphas, and alpha_bars. The space required is proportional to $T$ .

solution.py

Python 3.13

Test results