Camera Projection Matrix

1. Background Knowledge

In the standard pinhole camera model, a 3D point in homogeneous world coordinates is written as $(X, Y, Z, 1)^T$. The camera is modeled by a projection matrix $P$ of size $3 \times 4$, which encapsulates both intrinsic parameters (focal length, principal point, skew) and extrinsic parameters (rotation and translation from world to camera coordinates). Applying $P$ to a 3D homogeneous point gives a 2D homogeneous image point $(u, v, w)^T$.

Homogeneous coordinates let us express perspective projection as a linear transform followed by a divide. The key idea: linear algebra handles the projective mapping $P \cdot (X, Y, Z, 1)^T = (u, v, w)^T$, and then we convert back to standard 2D coordinates by normalizing: $(u', v') = (u/w, v/w)$. The division by $w$ produces the non-linear perspective effect: points further in depth map closer together in image space.

2. Algorithm / Approach

General pattern for this type of problem:

• Represent the 3D point in 4D homogeneous form: $(X, Y, Z, 1)$.
• Perform a matrix-vector multiplication with the $3 \times 4$ projection matrix $P$.
• Obtain a 3D homogeneous image point $(u, v, w)$.
• Convert to 2D inhomogeneous coordinates by dividing by $w$: $(u/w, v/w)$.

This is a straightforward application of linear algebra and homogeneous coordinate normalization.

3. Step-by-Step Strategy

Assume:

• P is a 3×4 matrix
• X, Y, Z are the world coordinates (scalars or elements of a vector)

Steps:

1. Form the input vector Create a 4D column vector for the 3D point in homogeneous form:

X_world = [X, Y, Z, 1] # or as a column vector

2. Multiply by the projection matrix Compute:

u = P*X + P*Y + P*Z + P*1
v = P*X + P*Y + P*Z + P*1
w = P*X + P*Y + P*Z + P*1

or using a library (e.g., NumPy):

import numpy as np
Xh = np.array([X, Y, Z, 1.0])
uh, vh, wh = P @ Xh # 3x4 @ 4 -> 3

3. Normalize homogeneous coordinates

u_img = u / w
v_img = v / w

4. Return the 2D point The final 2D image coordinates are:

return (u_img, v_img)

4. Common Pitfalls

• Forgetting the homogeneous 1: Using $(X, Y, Z)$ instead of $(X, Y, Z, 1)$ will make the multiplication invalid or incorrect.
• Not dividing by w: Returning $(u, v)$ directly from the matrix multiplication without normalizing by $w$ gives homogeneous, not pixel, coordinates.
• Division by zero or near-zero w: If $w = 0$ (or extremely small), the point is at or beyond the camera’s projection plane, and the projection is undefined or numerically unstable. You typically need to check abs(w) < eps.
• Type / precision issues: Using integer arithmetic can cause truncation during division; make sure to use floating-point types.
• Matrix shape mismatch: Ensure P is 3×4 and the point vector is length 4; otherwise, you’ll get runtime errors.

5. Time & Space Complexity

• Time complexity: For a single point, multiplying a $3 \times 4$ matrix by a 4D vector is $O(1)$ (a fixed number of multiplications and additions). For $N$ points, doing this independently is $O(N)$.

• Space complexity:

• Storing the projection matrix: $O(1)$ (12 numbers).

• Storing each point and its projection: $O(1)$ per point; $O(N)$ if you keep all $N$ results.