Disparity-Based View Warping

mediumPROView InterpolationIBRCV: Image-Based Rendering

Warp an image using a disparity map for view synthesis.

To synthesize a novel view at position $t$ between two cameras, we shift pixels based on their disparity:

$u_{src} = u_{tgt} - t \cdot d(u, v)$

where:

$u_{tgt}$ is the target pixel x-coordinate
$d(u, v)$ is the disparity at that pixel
$t$ is the interpolation factor (0 = left view, 1 = right view)
$u_{src}$ is where to sample from the source image

This "backward warping" approach samples the source image at computed coordinates, avoiding holes in the output.

Example

Input

warp_by_disparity([[1, 2, 3, 4]], [[0, 1, 1, 0]], 1)

Output

[[1, 1, 2, 4]]

Reasoning

Inputs: image is a $1 \times 4$ strip of intensities $[1, 2, 3, 4]$ ; disparity gives each target pixel's horizontal shift, $[0, 1, 1, 0]$ , so the two middle pixels belong to a nearer surface; t $= 1$ means we synthesise the full right-hand view.
Math: every output pixel $u_{tgt}$ asks where it came from, $u_{src} = u_{tgt} - t \cdot d$ . Only the column moves; the row stays the same.
Column 0: $0 - 1 \cdot 0 = 0$ , value $1$ . Column 1: $1 - 1 \cdot 1 = 0$ , value $1$ . Column 2: $2 - 1 \cdot 1 = 1$ , value $2$ . Column 3: $3 - 1 \cdot 0 = 3$ , value $4$ .
A fractional $u_{src}$ goes to the nearest column, and anything left of column 0 or right of column $w-1$ is clamped to the edge.
Output: [[1, 1, 2, 4]], same size as the input. Notice that intensity $3$ vanishes and $1$ appears twice: backward warping never leaves a hole, but it can duplicate or drop source pixels where disparity changes.

Constraints

image: 2D source image
disparity: 2D disparity map (same dimensions)
t: interpolation factor
Use nearest-neighbor sampling with boundary clamping
Return warped image

Learn this in

Image-Based RenderingCV Study Plan

More from CV: Image-Based Rendering

Differentiable Renderer Forwardhard Video Loop Findinghard Light Field Refocusinghard Depth-Based View Synthesishard Front-to-Back Splat Accumulationmedium

Concept primerPREVIEW

To solve this problem, you need to understand disparity, view interpolation, and backward warping for resampling images.

1. Background Knowledge

In a rectified stereo setup, corresponding pixels between left and right images lie on the same scanline (same $v$ ) but different horizontal positions $u$ . The disparity $d(u, v)$ measures this horizontal shift: for a point seen at $u_L$ in the left view and $u_R$ in the right view, disparity is typically $d = u_L - u_R$ (or the opposite, depending on convention). Disparity is inversely related to depth: closer objects have larger disparity.

View interpolation aims to render an intermediate view at a virtual camera position between the two real cameras. If we parameterize the camera position by $t \in [0, 1]$ (0 = left, 1 = right), pixels must be shifted horizontally according to disparity and $t$ . Instead of “pushing” each source pixel to its new location (forward warping, which causes holes), backward warping chooses each target pixel $(u_{tgt}, v)$ and computes where to sample in the source image:

u_{src} = u_{tgt} - t \cdot d(u_{tgt}, v)

You then read the color at $(u_{src}, v)$ in the source image with interpolation. This samples the source densely and avoids most holes.

2. Algorithm / Approach

The general pattern for disparity-based backward warping:

For each pixel $(u_{tgt}, v)$ in the output (target) image:

Look up disparity $d(u_{tgt}, v)$ .
Compute the corresponding source coordinate $u_{src}$ using the given formula.

Sample the source image at $(u_{src}, v)$ (using interpolation if non-integer).
Write that color into the target pixel.

You can use:

One source image (e.g., left or right) plus a disparity map defined in the target view.
Or, more generally, warp both left and right images toward the intermediate view and then blend or choose between them; but the core warping step is the same.

3. Step-by-Step Strategy

Assuming:

src is the source image (e.g., left view),
disp is the disparity map aligned with the target view grid,
t is the interpolation factor,
output shape is same as src:

Initialize output

Create an empty image out with the same height and width as the disparity map.

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results