📘

Bilinear Flow Warping

HardInterpolation 📚 CV: Motion Estimation

Implement frame warping using optical flow with bilinear interpolation to create an intermediate frame. This task involves understanding how to apply optical flow to warp a source frame.

The concept of optical flow is crucial here, as it represents the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer and the scene. Given the optical flow $(u, v)$ at each pixel, the goal is to warp the source frame to create an intermediate frame. The warping process involves shifting each pixel in the source frame by the corresponding optical flow values.

Here are the steps to achieve this:

For each output pixel $(x, y)$ , calculate the source location by adding the optical flow values $(u, v)$ .
Check if the calculated source location falls between pixels.
If it does, apply bilinear interpolation to determine the intensity value of the output pixel.

I_{out}(x, y) = I_{src}(x + u(x,y), y + v(x,y))

This technique is widely used in video processing applications, such as frame interpolation and video slow-motion.

Example:

Input:

src_frame = image at t=0
flow = motion from t=0 to t=1
warp_factor = 0.5

Output:

Intermediate frame at t=0.5

Reasoning:

For each pixel in output:

Look up flow at that position
Scale flow by warp_factor
Sample source at (x + u0.5, y + v0.5)
Bilinear interpolate if non-integer

Constraints:

src_frame: Source frame (H, W)
flow_u, flow_v: Optical flow fields (H, W)
Return: Warped frame

More from CV: Motion Estimation

Hungarian Algorithm for MOTHard Rolling Shutter CorrectionHard Horn-Schunck Global Optical FlowMedium Intersection over Union (IoU) for TrackingMedium Trajectory Smoothing FilterMedium

Background Knowledge

Frame interpolation is about synthesizing new frames between existing video frames to create smoother motion, e.g., for slow-motion or frame rate up-conversion. A common classical approach uses optical flow, which estimates per-pixel motion vectors between frames. Each vector $(u(x,y), v(x,y))$ indicates where a pixel at $(x,y)$ moves to in the next (or previous) frame.

To generate an intermediate frame, you typically warp one or more source frames: for each pixel in the output frame, you sample from a source frame at a displaced (non-integer) coordinate $(x+u, y+v)$ . Since images are defined on an integer grid, you need interpolation to get an intensity value at non-integer positions. Bilinear interpolation is the standard choice: it interpolates over the 2×2 neighborhood around the floating-point coordinate and linearly blends the four pixels.

Here, the problem is the core low-level operation: given a source image and a dense flow field $(u,v)$ , compute the warped image

I_{\text{out}}(x,y) = I_{\text{src}}(x + u(x,y),\, y + v(x,y))

using bilinear interpolation whenever $(x+u, y+v)$ is not on integer coordinates. This is often called backward warping (or inverse warping), because you iterate over target pixels and pull values from the source image using the flow.

Algorithm / Approach

General pattern (backward warping with bilinear interpolation):

For each integer pixel $(x,y)$ in the output image:
Use the flow to compute the corresponding continuous coordinate in the source: $(x_s, y_s) = (x + u(x,y),\, y + v(x,y))$ .
If $(x_s, y_s)$ is outside the source image, handle via a chosen border policy (e.g., skip, clamp, or fill with 0).
Otherwise:
Find the integer neighbors around $(x_s, y_s)$ .
Compute bilinear interpolation weights.
Linearly combine the 4 neighbor pixels with those weights to get $I_{$ \text{out} $}(x,y)$ .

This pattern is the same used in many libraries’ “grid_sample” / “remap” / “warp” operations.

Step-by-Step Strategy

Assume:

Source image size: height H, width W, possibly channels C.
Flow fields u[y][x], v[y][x] defined for each pixel in the output grid.

Step 1: Loop over output pixels

for y in range(H):
 for x in range(W):
 ux = u[y, x]
 vy = v[y, x]
 xs = x + ux
 ys = y + vy
 #... interpolate from src at (xs, ys)

Step 2: Boundary check

👑

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

🔒

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.