Horn-Schunck Global Optical Flow

mediumPROOptical FlowCVCV: Motion Estimation

Implement Horn-Schunck optical flow with smoothness constraint to estimate the motion of pixels between two consecutive frames. This method is used to calculate the optical flow, which is the apparent motion of pixels in an image.

The Horn-Schunck method assumes that the brightness constancy of the scene is maintained between frames, which can be expressed as $I_x u + I_y v + I_t = 0$ , where $I_x$ and $I_y$ are the spatial gradients of the image intensity, $I_t$ is the temporal gradient, and $u$ and $v$ are the horizontal and vertical components of the optical flow. To handle the aperture problem, a smoothness constraint weighted by $\alpha^2$ (where $\alpha$ is the alpha argument) is added to the energy function.

Here are the steps to calculate the optical flow:

Compute the spatial and temporal gradients of the image intensity.
Initialize the optical flow components $u$ and $v$ .
Iterate to minimize the energy function.

E = \int\int (I_x u + I_y v + I_t)^2 + \alpha^2(|\nabla u|^2 + |\nabla v|^2) dx dy

This technique is widely used in video analysis and object tracking applications.

Example

Input

Two frames

Output

Dense optical flow

Reasoning

Inputs: img1, img2 are consecutive $H \times W$ grayscale frames; alpha weights smoothness, n_iter counts sweeps. The test uses two all-zero $16 \times 16$ frames, $\alpha = 1$ , 100 sweeps.
Gradients: $I_x$ is the horizontal (left-to-right) brightness change and $I_y$ the vertical (top-to-bottom) change in the first frame; $I_t = I_2 - I_1$ is the change between frames. Here all three are $0$ .
Math: each pixel's $(u, v)$ trades the brightness-constancy error $(I_x u + I_y v + I_t)^2$ against $\alpha^2$ times its deviation from the neighbours' average. At zero flow both terms vanish, so $u = v = 0$ after every sweep.
Output: dense optical flow, an $H \times W \times 2$ array (channel 0 is $u$ , channel 1 is $v$ ); the test prints (16, 16, 2).
Notice: flow is measurable only where there is texture. On a horizontal ramp ( $I_y = 0$ , constant $I_x$ ) the flow settles at $u = -I_t / I_x$ , $v = 0$ , and flat regions borrow flow from textured neighbours.

Constraints

Input frames: 2D numpy arrays (grayscale) with shape (H, W) and dtype float32
Pixel values: in range [0, 255] and assumed to be normalized to [0, 1] for computation
Output optical flow: 2D numpy array with shape (H, W, 2) and dtype float32, representing horizontal and vertical flow components
Smoothness parameter: α is a positive scalar, assumed to be provided as an input or set to a default value of 0.001
Iteration limit: maximum number of iterations to minimize the energy function, assumed to be set to a default value of 1000 or provided as an input

Learn this in

Motion EstimationCV Study Plan

More from CV: Motion Estimation

Hungarian Algorithm for MOThard Rolling Shutter Correctionhard Intersection over Union (IoU) for Trackingmedium Bilinear Flow Warpinghard Trajectory Smoothing Filtermedium

Horn-Schunck Global Optical Flow

Example

Constraints

Learn this in

More from CV: Motion Estimation

Background Knowledge

Continue the full explanation

Editor locked