📘

Reprojection Error

MediumOptimization 📚 CV: Image Alignment and Stitching

Implement a function to calculate the reprojection error, a crucial component in bundle adjustment, which is a key technique in image alignment and stitching. The reprojection error measures the difference between the projected location of a 3D point and its observed location in a 2D image.

The concept of reprojection error is rooted in the pinhole camera model, where a 3D point $\mathbf{X}$ is transformed into a 2D point $(u, v)$ using the camera's rotation matrix $R$ and translation vector $\mathbf{t}$ . This transformation is followed by a perspective projection, which maps the 3D point to a 2D point using the camera's focal length $f$ .

To compute the reprojection error, the following steps are involved:

Transform the 3D point using the rotation matrix and translation vector: $\mathbf{X}' = R\mathbf{X} + \mathbf{t}$
Project the transformed 3D point to a 2D point: $(u, v) = \left(f\frac{X'}{Z'}, f\frac{Y'}{Z'}\right)$
Calculate the error between the projected 2D point and the observed 2D point

e = (u - u_{obs})^2 + (v - v_{obs})^2

This technique is widely used in computer vision applications, such as structure from motion and stereo vision.

Example:

Input:

X = [0, 0, 10]
observed = [0, 0]
R = [[1,0,0], [0,1,0], [0,0,1]]
t = [0, 0, 0]
focal = 1

Output:

0.0

Reasoning:

Apply rotation (identity) and translation (zero): X' = RX + t = [0, 0, 10] + [0, 0, 0] = [0, 0, 10]
Project to 2D: u = focal × X' / Z' = 1 × 0 / 10 = 0 v = focal × Y' / Z' = 1 × 0 / 10 = 0 Projected: (0, 0)
Compute error: e = (0 - 0)² + (0 - 0)² = 0

The 3D point projects exactly to the observed location.

Constraints:

X is 3D point [X, Y, Z]
observed is 2D point [u, v]
R is 3x3 rotation matrix (use identity for simplicity)
t is translation [tx, ty, tz]
focal is focal length
Return squared reprojection error rounded to 4 decimal places

More from CV: Image Alignment and Stitching

Jacobian for Bundle AdjustmentHard Cylindrical Projection for PanoramasHard Essential Matrix EstimationHard Procrustes AnalysisMedium Gradient Descent UpdateMedium

You are computing a single reprojection error term: how far a 3D point, when projected with a given camera, is from where it was actually observed in the image. This is the basic building block of the bundle adjustment objective.

1. Background Knowledge

In pinhole camera geometry, a 3D point \mathbf{X} $= (X, Y, Z)^\top$ in world (or scene) coordinates is mapped into camera coordinates using the camera pose: a rotation $R$ and translation $\mathbf{t}$ . This gives the point as seen from the camera’s coordinate frame:

\mathbf{X}' = (X', Y', Z')^\top = R\mathbf{X} + \mathbf{t}.

Then, under a simple pinhole model with a single focal length $f$ and no principal point offset or distortion, the point is projected onto the image plane as:

u = f \frac{X'}{Z'}, \quad v = f \frac{Y'}{Z'}.

Here $(u, v)$ are the predicted image coordinates of the 3D point.

The reprojection error compares this predicted location to the observed 2D feature location $(u_{obs}, v_{obs})$ . In least-squares form:

e = (u - u_{obs})^2 + (v - v_{obs})^2,

which is just the squared Euclidean distance between predicted and observed points. Bundle adjustment sums these errors over many points and cameras and minimizes the total to refine both 3D structure and camera parameters.

2. Algorithm / General Approach

The general pattern for these problems is:

Apply the camera pose to move the 3D point into camera coordinates.
Project the 3D point from camera coordinates onto the image plane using the camera intrinsics (here just focal length).
Compute the residual between this projection and the observed 2D point.
Form the error as the squared norm of this residual.

In code, this is typically a few linear algebra operations and basic arithmetic.

3. Step-by-Step Strategy

Assume you are given:

3D point: \mathbf{X} $= (X, Y, Z)$
Rotation matrix: $R \in \mathbb{R}^{3 \times 3}$
Translation vector: \mathbf{t} $= (t_x, t_y, t_z)$
Focal length: $f$
Observed 2D point: $(u_{obs}, v_{obs})$

Steps:

👑

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

🔒

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.