Volume Rendering Weights

mediumFREENeural 3D RepresentationsNeural 3DCV: 3D Reconstruction

Compute volume rendering weights along a ray for NeRF.

NeRF renders images by accumulating color along rays. The weight at each sample point is:

$w_i = T_i \cdot \alpha_i$

where:

$\alpha_i = 1 - \exp(-\sigma_i \cdot \delta_i)$ is the opacity
$T_i = \exp\left(-\sum_{j<i} \sigma_j \cdot \delta_j\right)$ is the transmittance
$\sigma_i$ is the density at sample $i$
$\delta_i$ is the step size (distance to next sample)

The weight represents how much each sample contributes to the final pixel color.

Example

Input

volume_weights([1, 1, 1], [1, 1, 1])

Output

[0.6321, 0.2325, 0.0855]

Reasoning

Inputs: densities $\sigma = [1, 1, 1]$ and deltas $\delta = [1, 1, 1]$ describe three samples along one ray, each with density 1 over a segment of length 1, so every product $\sigma_i \delta_i = 1$ .
Opacity is identical for all three samples: $\alpha_i = 1 - e^{-1} \approx 0.6321$ .
Transmittance is the light that survives every earlier segment: $T_0 = e^{0} = 1$ (nothing in front of it), $T_1 = e^{-1} \approx 0.3679$ , $T_2 = e^{-(1+1)} = e^{-2} \approx 0.1353$ .
Weights $w_i = T_i \alpha_i$ : $w_0 = 1 \times 0.6321 = 0.6321$ , $w_1 = 0.3679 \times 0.6321 \approx 0.2325$ , $w_2 = 0.1353 \times 0.6321 \approx 0.0855$ , giving [0.6321, 0.2325, 0.0855] (shown rounded to 4 decimal places).
Notice the weights shrink even though the samples are identical: later samples sit behind earlier ones. They sum to $1 - e^{-3} \approx 0.9502$ , not 1, because $e^{-3}$ of the light passes through the whole ray. With all densities 0, every $\alpha_i = 0$ and every weight is 0.

Constraints

densities: list of σ values at each sample
deltas: list of step sizes between samples
Return list of weights

Learn this in

3D ReconstructionCV Study Plan Image-Based RenderingCV Study Plan Neural Radiance Fields (NeRF)3D Computer Vision

More from CV: 3D Reconstruction

NeRF Ray Samplinghard Surface Integration from Normalsmedium Marching Cubes SDF to Meshmedium ICP Point Cloud Alignmenthard NeRF Positional Encodingmedium

Concept primer

To solve this problem, you need to understand how NeRF turns continuous volume rendering into a discrete, differentiable weighting scheme along each camera ray.

1. Background Knowledge

In NeRF, a 3D scene is modeled as a continuous radiance field: a function that maps a 3D point and viewing direction to density $\sigma$ and color. Rendering an image means, for each pixel, casting a ray into the scene and integrating color along that ray, weighted by how much light is not absorbed before reaching each point.

Classical volume rendering (from graphics) defines the pixel color as an integral of emitted radiance times transmittance (probability that light travels from the camera to that point without being blocked) and opacity at each depth. NeRF discretizes this integral by sampling points along the ray and computing a weight $w_i$ for each sample. These weights determine how much each sample contributes to the final pixel color and sum to at most 1.

The given formulas:

$\alpha_i = 1 - \exp(-$ \sigma $_i$ \delta $_i)$ (opacity of segment $i$ )
$T_i = \exp(-$ \sum_{j<i}\sigma $_j$ \delta $_j)$ (transmittance up to sample $i$ )
$w_i = T_i$ \alpha $_i$

are the discrete approximation of that integral. Your task is to compute $w_i$ efficiently and stably for all samples along a ray.

2. Algorithm / General Approach

This is a prefix-product / cumulative-transmittance computation with exponentials:

You are given arrays:
densities $\sigma_i$ (shape e.g. [N] or [num_rays, num_samples])
step sizes $\delta_i$ (same shape, usually distances to next sample)
First, compute $\alpha_i$ from $\sigma_i$ and $\delta_i$ .
Then compute cumulative absorption $\sum_{j<i}$ \sigma $_j$ \delta $_j$ (a prefix sum).
Turn that into transmittance $T_i = \exp(-$ \text{prefix_sum} $)$ .
Finally, compute weights $w_i = T_i$ \alpha $_i$ .

The pattern is: elementwise transform → prefix sum → elementwise exponential → elementwise product.

3. Step-by-Step Strategy

Assume 1 ray with $N$ samples; extension to many rays is done per-ray independently.

Inputs

Array sigma[i] for $i = 0,\dots,N-1$
Array delta[i] for $i = 0,\dots,N-1$

Compute segment optical thickness

tau = sigma * delta # tau[i] = sigma_i * delta_i

Compute opacity $\alpha_i$

alpha = 1.0 - torch.exp(-tau)
# or np.exp() if using NumPy

Compute cumulative absorption before each sample

You want:

prefix sum over tau but excluding the current index $i$ , i.e. cumsum_exclusive[i] = sum_{j < i} tau[j].

Example pattern:

cumsum = torch.cumsum(tau, dim=-1) # inclusive: sum_{j <= i}
# exclusive by shifting:
# T_0 should use sum over empty set = 0
cumsum_exclusive = torch.cat(
[torch.zeros_like(cumsum[..., :1]), cumsum[..., :-1]],
dim=-1
)

Compute transmittance $T_i$

T = torch.exp(-cumsum_exclusive)

Compute weights $w_i$

weights = T * alpha

(Optionally) Compute final pixel color if you also have colors rgb[i]:

# rgb: shape (..., N, 3)
color = torch.sum(weights[..., None] * rgb, dim=-2)

4. Common Pitfalls

Off-by-one error in prefix sum: Make sure $T_0$ uses an empty sum (i.e., $T_0 = 1$ ), and for general $i$ , the sum only includes indices < i, not <= i.
Numerical stability
Large sigma * delta can cause exp(-tau) to underflow to 0; this is usually fine, but avoid NaNs.
Clamp sigma or tau if needed, or ensure they remain finite.
In some implementations, T can be accumulated via a multiplicative form like:

T = torch.cumprod(torch.cat([torch.ones_like(alpha[..., :1]),
1.0 - alpha + eps], dim=-1), dim=-1)[..., :-1]

which avoids repeated exponentials but uses products of (1 - alpha).

Broadcasting / shape issues: When working with many rays, ensure dimensions match, e.g. (num_rays, num_samples) and that prefix sums are taken along the sample dimension only.
Last interval $\delta_N$ : Often delta is defined only between samples; for the last sample, some code uses a large value or repeats the previous step. Be consistent with how your input is defined.

5. Time & Space Complexity

Let:

$R$ = number of rays
$N$ = number of samples per ray

Per ray:

Computing tau, alpha: $O(N)$
Prefix sum (cumsum): $O(N)$
Exponentials and final products: $O(N)$

So:

Time complexity: $O(RN)$
Space complexity:
Storing sigma, delta, alpha, T, weights: all $O(RN)$ .
If you reuse buffers or compute in-place where allowed, you can keep extra overhead to $O(RN)$ total.

solution.py

Python 3.13

Test results