Brightness Constancy Residual

To solve this problem, you just need to apply the optical flow constraint at a single pixel and compute its scalar residual.

---

1. Background Knowledge

Optical flow describes the apparent motion of pixels between consecutive frames in a video. For a pixel at location $(x, y)$ in frame $t$, its motion to frame $t+1$ is given by a flow vector $(u, v)$, where $u$ is the horizontal displacement and $v$ is the vertical displacement.

A key modeling assumption is brightness constancy: the intensity of a moving point in the scene stays (approximately) the same over a short time:

$$I(x, y, t) \approx I(x+u, y+v, t+1)$$

If you linearize this with a first-order Taylor expansion, you get the optical flow constraint equation:

$$I_x u + I_y v + I_t = 0$$

where $I_x, I_y$ are spatial derivatives and $I_t$ is the temporal derivative (intensity change between frames). In practice this is rarely exactly zero, so we talk about a residual

$$r = I_x u + I_y v + I_t$$

as a measure of how well a candidate flow $(u, v)$ satisfies brightness constancy at that pixel.

---

2. Algorithm / Approach

For this coding problem, you are not estimating flow; you are evaluating how good a given flow vector is at a single pixel.

The pattern is:

1. You are given:

• The derivatives $I_x, I_y, I_t$ at one pixel.
• The optical flow vector $(u, v)$ at that pixel.

2. Plug them into the brightness constancy constraint.
3. Return the scalar residual $r$.

Computationally, this is just a dot product:

$$r = [I_x, I_y] \cdot [u, v] + I_t$$

---

3. Step-by-Step Strategy

Assume inputs (in any order your platform uses) are: Ix, Iy, It, u, v.

1. Read the inputs:

• Spatial derivatives: Ix, Iy
• Temporal derivative: It
• Flow vector components: u, v

2. Compute the linear combination:

r = Ix * u + Iy * v + It

3. Return or print r.

If the problem uses arrays or multiple pixels, you would apply this operation elementwise, but for a single pixel it’s just scalar arithmetic.

---

4. Common Pitfalls

• Sign mistakes: The residual is $r = I_x u + I_y v + I_t$, not $I_x u + I_y v - I_t$. Be careful to match the definition given in the prompt.

• Wrong variable order: Ensure you know which input corresponds to $I_x, I_y, I_t, u, v$. Mis-ordering them will silently give wrong results.

• Data types / precision: Use floating-point arithmetic (float or double) since derivatives and flow are rarely integers.

• Misinterpreting the goal: You are computing the residual, not trying to adjust $(u, v)$ to make it zero.

---

5. Time & Space Complexity

For a single pixel:

• Time complexity: Constant, $O(1)$, since it’s a fixed number of multiplications and additions.

• Space complexity: Constant, $O(1)$, as you only store a handful of scalar values.

If extended to an image with $N$ pixels, both time and space would scale linearly, $O(N)$, when computing one residual per pixel.