PIXELBANKv8.2.1
Menu
Sign In
Back to CV Study Plan
Week 1-2

Chapter 2: Image Formation

Understanding how images are formed through geometric transformations, camera projections, the physics of light interaction with surfaces, and how digital cameras capture the visual world.

Chapter Overview

How does a 3D world become a 2D image? This fundamental question is at the heart of computer vision.

What is this chapter about? We study the complete image formation pipeline—from geometric transformations that describe how objects move in space, to camera models that project 3D points onto 2D images, to the physics of how light interacts with surfaces.

Why does this matter? Understanding image formation is essential for:

  • 3D reconstruction: Inverting the projection to recover 3D from 2D
  • Camera calibration: Measuring camera parameters for accurate measurements
  • Augmented reality: Placing virtual objects correctly in real scenes
  • Multi-view geometry: Understanding relationships between multiple views

How the topics connect: We start with geometric transformations in 2D and 3D—the mathematics of position and motion. Then we model cameras as projection systems that map 3D to 2D. We explore how light and color form the actual pixel values. Finally, we understand how digital sensors sample this continuous signal into discrete pixels.

Chapter Roadmap

Click any topic to jump in

1
2D/3D Transforms

Homogeneous coordinates, rotation matrices, and composition — geometric transformations in 2D and 3D.

Cameras and lenses

Projection and distortion

2
3D to 2D Projection

Camera intrinsics, extrinsics, perspective vs orthographic — how 3D worlds become 2D images.

3
Lens Distortion

Radial and tangential distortion, undistortion — correcting real camera imperfections.

Appearance modeling

Light and color

4
Lighting & Reflectance

Lambertian, specular, Phong, and BRDF models — how light interacts with surfaces.

5
Color & Camera

Bayer patterns, color spaces, white balance, gamma — how cameras capture color.

Signal processing constraints
6
Sampling & Aliasing

Nyquist theorem, aliasing, anti-aliasing, and image pyramids — the signal processing foundation of CV.

Everything in computer vision involves transformations—moving cameras, rotating objects, warping images. This topic builds your fluency with the mathematical language of geometric transformations.

We'll see how homogeneous coordinates unify all transformations into matrix multiplications, making it elegant to compose multiple operations. This foundation is used throughout CV for camera models, image warping, and 3D reconstruction.

Definition

Geometric transformations map points from one coordinate system to another. In computer vision, these transformations are fundamental for modeling camera motion, object pose, image warping, and augmented reality applications.

The key transformations are:

  • Translation: Shifting all points by a fixed offset
  • Rotation: Rotating points around an axis
  • Scaling: Changing the size of objects
  • Affine: Preserves parallel lines (includes shear)
  • Projective: Most general, used for camera models

In this topic

1Homogeneous Coordinates
2Rotation Matrix
33D Rotations
4Transformation Composition
1 of 4
Homogeneous Coordinates

(x′y′1)=(abtxcdty001)(xy1)\begin{pmatrix} x' \\ y' \\ 1 \end{pmatrix} = \begin{pmatrix} a & b & t_x \\ c & d & t_y \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \\ 1 \end{pmatrix}

Standard 2D transformations like rotation and scaling can be represented as 2×2 matrix multiplications. However, translation cannot—it requires addition. Homogeneous coordinates solve this by adding an extra dimension:

  • 2D point (x,y)(x, y) becomes (x,y,1)(x, y, 1)
  • Now ALL transformations (including translation) can be expressed as matrix multiplications
  • This allows us to chain transformations by multiplying matrices

The extra coordinate ww is typically 1. To convert back: (x,y)=(x~/w,y~/w)(x, y) = (\tilde{x}/w, \tilde{y}/w)

Points at Infinity: When w=0w = 0, we get points at infinity—useful for representing directions and vanishing points in perspective geometry.

Transformation Hierarchy:

  • Euclidean (3 DOF): Rotation + translation, preserves distances
  • Similarity (4 DOF): + uniform scale, preserves angles
  • Affine (6 DOF): General 2×3 matrix, preserves parallel lines
  • Projective (8 DOF): Full 3×3 matrix, only preserves straight lines
Mathematical Intuition

Appending a coordinate turns the Euclidean group into a matrix group: [x,y,1]T[x, y, 1]^T in P2\mathbb{P}^2 lets translations become linear. The projective equivalence λp~∼p~\lambda\tilde{\mathbf{p}} \sim \tilde{\mathbf{p}} is what makes perspective projection a matrix multiply — dividing by the third coordinate recovers pixel positions from homogeneous vectors.

Example:

Translate point (5,3)(5, 3) by (10,−2)(10, -2) using a homogeneous transformation matrix.

2 of 4
Rotation Matrix

R(θ)=(cos⁡θ−sin⁡θsin⁡θcos⁡θ)R(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

A 2D rotation by angle θ\theta (counterclockwise) is represented by an orthogonal matrix with determinant 1. Key properties:

  • RT=R−1R^T = R^{-1} (inverse is transpose)
  • det⁡(R)=1\det(R) = 1 (preserves area, proper rotation)
  • det⁡(R)=−1\det(R) = -1 (improper rotation, includes reflection)
  • Columns are orthonormal vectors (form an orthonormal basis)
  • Composing rotations: Rtotal=R2⋅R1R_{total} = R_2 \cdot R_1

The Special Orthogonal Group SO(2): All 2D rotation matrices form a mathematical group called SO(2). This has important implications for interpolating between rotations and understanding motion.

Small Angle Approximation: For small θ\theta (radians): R≈(1−θθ1)R \approx \begin{pmatrix} 1 & -\theta \\ \theta & 1 \end{pmatrix} Useful for linearizing rotation in optimization.

Mathematical Intuition

Rotation matrices live in SO(2): orthogonal (RTR=I\mathbf{R}^T\mathbf{R} = \mathbf{I}) with det⁡(R)=1\det(\mathbf{R}) = 1. The orthogonality constraint means R−1=RT\mathbf{R}^{-1} = \mathbf{R}^T, so inverting a rotation is free — just transpose. The single-parameter family R(θ)\mathbf{R}(\theta) forms a group: composing two rotations is adding angles, R(α)R(β)=R(α+β)\mathbf{R}(\alpha)\mathbf{R}(\beta) = \mathbf{R}(\alpha + \beta).

Example:

Rotate the unit vector (1,0)(1, 0) by 30°30° counterclockwise.

3 of 4
3D Rotations

Rx=(1000cos⁡θ−sin⁡θ0sin⁡θcos⁡θ)R_x = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \cos\theta & -\sin\theta \\ 0 & \sin\theta & \cos\theta \end{pmatrix}

In 3D, rotations are more complex and form the group SO(3) (3 degrees of freedom despite 9 matrix elements). Common representations:

  1. Rotation Matrices: 3×3 orthogonal matrices with det=1. Easy to compose but have 9 params for 3 DOF.

  2. Euler Angles: Three sequential rotations (e.g., roll-pitch-yaw). Intuitive but suffer from gimbal lock where you lose a degree of freedom.

  3. Axis-Angle: Rotation by angle θ\theta around unit axis n\mathbf{n}. Rodrigues' formula converts to matrix: R=I+(sin⁡θ)K+(1−cos⁡θ)K2R = I + (\sin\theta)K + (1-\cos\theta)K^2 where KK is the skew-symmetric matrix of n\mathbf{n}.

  4. Quaternions: 4D unit vectors q=(w,x,y,z)q = (w, x, y, z) with ∣q∣=1|q|=1. Best for interpolation (SLERP) and avoids gimbal lock. Standard in robotics and 3D vision.

Lie Algebra: The tangent space of SO(3) at identity, denoted so(3)\mathfrak{so}(3), is the space of skew-symmetric matrices. This is key for optimization on rotations.

Mathematical Intuition

3D rotations live in SO(3), a 3-dimensional manifold parameterized by 3 angles but embedded in R3×3\mathbb{R}^{3 \times 3} with 6 constraints (RTR=I\mathbf{R}^T\mathbf{R} = \mathbf{I}). Unlike 2D rotations, 3D rotations do not commute: RxRy≠RyRx\mathbf{R}_x\mathbf{R}_y \neq \mathbf{R}_y\mathbf{R}_x. This non-commutativity is why rotation order matters and why Euler angles suffer from gimbal lock.

Example:

What rotation matrix rotates 90°90° around the z-axis (yaw)?

4 of 4
Transformation Composition

Multiple transformations can be combined by matrix multiplication. The order matters—transformations are applied right-to-left:

M=T⋅R⋅SM = T \cdot R \cdot S means: first Scale, then Rotate, then Translate

Rotating Around a Point P:

  1. Translate P to origin: T−PT_{-P}
  2. Rotate: RR
  3. Translate back: TPT_P

Combined: M=TP⋅R⋅T−PM = T_P \cdot R \cdot T_{-P}

Inverse Transformations: To undo M=T⋅R⋅SM = T \cdot R \cdot S: M−1=S−1⋅R−1⋅T−1=S−1⋅RT⋅T−1M^{-1} = S^{-1} \cdot R^{-1} \cdot T^{-1} = S^{-1} \cdot R^T \cdot T^{-1}

Camera Transforms: World → Camera uses extrinsics [R∣t][R|t], then Camera → Image uses intrinsics KK.

Mathematical Intuition

Composition is matrix multiplication: applying transform A\mathbf{A} then B\mathbf{B} gives BA\mathbf{B}\mathbf{A} (right-to-left). This associativity means you can precompute M=BA\mathbf{M} = \mathbf{B}\mathbf{A} once and apply it to millions of points — the cost is one matrix multiply per point regardless of how many transforms were composed.

Example:

To rotate around point (100,100)(100, 100) by 45°45°, what sequence of transforms is needed?

Theory Exercise

Problem:

A robot arm needs to:

  1. Rotate 45° counterclockwise
  2. Then translate by (10, 5)

Write the combined transformation matrix in homogeneous coordinates. Then apply it to the point (2, 0) and find the final position.

Hints:
  • Remember: apply rotation first, then translation
  • For 45°: cos(45°) = sin(45°) = √2/2 ≈ 0.707
  • The combined matrix is T · R
Blogs
Homogeneous Coordinates — Song Ho
Previous
Introduction
Next
Image Processing