Week 1-2

Chapter 2: Image Formation

Understanding how images are formed through geometric transformations, camera projections, the physics of light interaction with surfaces, and how digital cameras capture the visual world.

Chapter Overview

How does a 3D world become a 2D image? This fundamental question is at the heart of computer vision.

What is this chapter about? We study the complete image formation pipeline—from geometric transformations that describe how objects move in space, to camera models that project 3D points onto 2D images, to the physics of how light interacts with surfaces.

Why does this matter? Understanding image formation is essential for:

3D reconstruction: Inverting the projection to recover 3D from 2D
Camera calibration: Measuring camera parameters for accurate measurements
Augmented reality: Placing virtual objects correctly in real scenes
Multi-view geometry: Understanding relationships between multiple views

How the topics connect: We start with geometric transformations in 2D and 3D—the mathematics of position and motion. Then we model cameras as projection systems that map 3D to 2D. We explore how light and color form the actual pixel values. Finally, we understand how digital sensors sample this continuous signal into discrete pixels.

Chapter Roadmap

Click any topic to jump in

2D/3D Transforms

Homogeneous coordinates, rotation matrices, and composition — geometric transformations in 2D and 3D.

Cameras and lenses

Projection and distortion

3D to 2D Projection

Camera intrinsics, extrinsics, perspective vs orthographic — how 3D worlds become 2D images.

Lens Distortion

Radial and tangential distortion, undistortion — correcting real camera imperfections.

Appearance modeling

Light and color

Lighting & Reflectance

Lambertian, specular, Phong, and BRDF models — how light interacts with surfaces.

Color & Camera

Bayer patterns, color spaces, white balance, gamma — how cameras capture color.

Signal processing constraints

Sampling & Aliasing

Nyquist theorem, aliasing, anti-aliasing, and image pyramids — the signal processing foundation of CV.

Everything in computer vision involves transformations—moving cameras, rotating objects, warping images. This topic builds your fluency with the mathematical language of geometric transformations.

We'll see how homogeneous coordinates unify all transformations into matrix multiplications, making it elegant to compose multiple operations. This foundation is used throughout CV for camera models, image warping, and 3D reconstruction.

Definition

Geometric transformations map points from one coordinate system to another. In computer vision, these transformations are fundamental for modeling camera motion, object pose, image warping, and augmented reality applications.

The key transformations are:

Translation: Shifting all points by a fixed offset
Rotation: Rotating points around an axis
Scaling: Changing the size of objects
Affine: Preserves parallel lines (includes shear)
Projective: Most general, used for camera models

In this topic

1Homogeneous Coordinates

2Rotation Matrix

33D Rotations

4Transformation Composition

1 of 4

Homogeneous Coordinates

$\begin{pmatrix} x' \ y' \ 1 \end{pmatrix} = \begin{pmatrix} a & b & t_x \ c & d & t_y \ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x \ y \ 1 \end{pmatrix}$

Standard 2D transformations like rotation and scaling can be represented as 2×2 matrix multiplications. However, translation cannot—it requires addition. Homogeneous coordinates solve this by adding an extra dimension:

2D point $(x, y)$ becomes $(x, y, 1)$
Now ALL transformations (including translation) can be expressed as matrix multiplications
This allows us to chain transformations by multiplying matrices

The extra coordinate $w$ is typically 1. To convert back: $(x, y) = (\tilde{x}/w, \tilde{y}/w)$

Points at Infinity: When $w = 0$ , we get points at infinity—useful for representing directions and vanishing points in perspective geometry.

Transformation Hierarchy:

Euclidean (3 DOF): Rotation + translation, preserves distances
Similarity (4 DOF): + uniform scale, preserves angles
Affine (6 DOF): General 2×3 matrix, preserves parallel lines
Projective (8 DOF): Full 3×3 matrix, only preserves straight lines

Mathematical Intuition

Appending a coordinate turns the Euclidean group into a matrix group: $[x, y, 1]^T$ in $\mathbb{P}^2$ lets translations become linear. The projective equivalence $\lambda\tilde{\mathbf{p}} \sim \tilde{\mathbf{p}}$ is what makes perspective projection a matrix multiply — dividing by the third coordinate recovers pixel positions from homogeneous vectors.

Example:

Translate point $(5, 3)$ by $(10, -2)$ using a homogeneous transformation matrix.

2 of 4

Rotation Matrix

$R(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta \ \sin\theta & \cos\theta \end{pmatrix}$

A 2D rotation by angle $\theta$ (counterclockwise) is represented by an orthogonal matrix with determinant 1. Key properties:

$R^T = R^{-1}$ (inverse is transpose)
$\det(R) = 1$ (preserves area, proper rotation)
$\det(R) = -1$ (improper rotation, includes reflection)
Columns are orthonormal vectors (form an orthonormal basis)
Composing rotations: $R_{total} = R_2 \cdot R_1$

The Special Orthogonal Group SO(2): All 2D rotation matrices form a mathematical group called SO(2). This has important implications for interpolating between rotations and understanding motion.

Small Angle Approximation: For small $\theta$ (radians): $R \approx \begin{pmatrix} 1 & -\theta \ \theta & 1 \end{pmatrix}$ Useful for linearizing rotation in optimization.

Mathematical Intuition

Rotation matrices live in SO(2): orthogonal ( $\mathbf{R}^T\mathbf{R} = \mathbf{I}$ ) with $\det(\mathbf{R}) = 1$ . The orthogonality constraint means $\mathbf{R}^{-1} = \mathbf{R}^T$ , so inverting a rotation is free — just transpose. The single-parameter family $\mathbf{R}(\theta)$ forms a group: composing two rotations is adding angles, $\mathbf{R}(\alpha)\mathbf{R}(\beta) = \mathbf{R}(\alpha + \beta)$ .

Example:

Rotate the unit vector $(1, 0)$ by $30°$ counterclockwise.

3 of 4

3D Rotations

$R_x = \begin{pmatrix} 1 & 0 & 0 \ 0 & \cos\theta & -\sin\theta \ 0 & \sin\theta & \cos\theta \end{pmatrix}$

In 3D, rotations are more complex and form the group SO(3) (3 degrees of freedom despite 9 matrix elements). Common representations:

Rotation Matrices: 3×3 orthogonal matrices with det=1. Easy to compose but have 9 params for 3 DOF.
Euler Angles: Three sequential rotations (e.g., roll-pitch-yaw). Intuitive but suffer from gimbal lock where you lose a degree of freedom.
Axis-Angle: Rotation by angle $\theta$ around unit axis $\mathbf{n}$ . Rodrigues' formula converts to matrix: $R = I + (\sin\theta)K + (1-\cos\theta)K^2$ where $K$ is the skew-symmetric matrix of $\mathbf{n}$ .
Quaternions: 4D unit vectors $q = (w, x, y, z)$ with $|q|=1$ . Best for interpolation (SLERP) and avoids gimbal lock. Standard in robotics and 3D vision.

Lie Algebra: The tangent space of SO(3) at identity, denoted $\mathfrak{so}(3)$ , is the space of skew-symmetric matrices. This is key for optimization on rotations.

Mathematical Intuition

3D rotations live in SO(3), a 3-dimensional manifold parameterized by 3 angles but embedded in $\mathbb{R}^{3 \times 3}$ with 6 constraints ( $\mathbf{R}^T\mathbf{R} = \mathbf{I}$ ). Unlike 2D rotations, 3D rotations do not commute: $\mathbf{R}_x\mathbf{R}_y \neq \mathbf{R}_y\mathbf{R}_x$ . This non-commutativity is why rotation order matters and why Euler angles suffer from gimbal lock.

Example:

What rotation matrix rotates $90°$ around the z-axis (yaw)?

4 of 4

Transformation Composition

Multiple transformations can be combined by matrix multiplication. The order matters—transformations are applied right-to-left:

$M = T \cdot R \cdot S$ means: first Scale, then Rotate, then Translate

Rotating Around a Point P:

Translate P to origin: $T_{-P}$
Rotate: $R$
Translate back: $T_P$

Combined: $M = T_P \cdot R \cdot T_{-P}$

Inverse Transformations: To undo $M = T \cdot R \cdot S$ : $M^{-1} = S^{-1} \cdot R^{-1} \cdot T^{-1} = S^{-1} \cdot R^T \cdot T^{-1}$

Camera Transforms: World → Camera uses extrinsics $[R|t]$ , then Camera → Image uses intrinsics $K$ .

Mathematical Intuition

Composition is matrix multiplication: applying transform $\mathbf{A}$ then $\mathbf{B}$ gives $\mathbf{B}\mathbf{A}$ (right-to-left). This associativity means you can precompute $\mathbf{M} = \mathbf{B}\mathbf{A}$ once and apply it to millions of points — the cost is one matrix multiply per point regardless of how many transforms were composed.

Example:

To rotate around point $(100, 100)$ by $45°$ , what sequence of transforms is needed?

Theory Exercise

Problem:

A robot arm needs to:

Rotate 45° counterclockwise
Then translate by (10, 5)

Write the combined transformation matrix in homogeneous coordinates. Then apply it to the point (2, 0) and find the final position.

Hints:

Remember: apply rotation first, then translation
For 45°: cos(45°) = sin(45°) = √2/2 ≈ 0.707
The combined matrix is T · R

Blogs

Homogeneous Coordinates — Song Ho

Introduction

Image Processing

Chapter 2: Image Formation

Chapter Overview

Chapter Roadmap

2D/3D Transforms

3D to 2D Projection

Lens Distortion

Lighting & Reflectance

Color & Camera

Sampling & Aliasing

2D and 3D Transformations

Definition

In this topic

Homogeneous Coordinates

Rotation Matrix

3D Rotations

Transformation Composition

Theory Exercise

Problem:

Hints:

3D to 2D Projections

Lens Distortions

Lighting and Reflectance

Color and the Digital Camera

Sampling and Aliasing