Loading...
Understand how cameras project the 3D world onto 2D images. Learn the pinhole model, lens distortion, intrinsic and extrinsic parameters, and practical calibration techniques using checkerboards and stereo rigs. These concepts are the bridge between physical imaging and the mathematical models every 3D vision algorithm depends on.
Camera models describe the mathematical relationship between 3D points in the world and their corresponding 2D pixel locations in an image. Without an accurate camera model, no 3D reconstruction, no stereo matching, and no augmented reality overlay can work correctly.
What is this chapter about? We build the complete camera pipeline from first principles: starting with the idealized pinhole model, adding real-world lens distortion, formalizing intrinsic and extrinsic parameters, and then learning how to estimate all of these from calibration images.
Why does this matter? Every algorithm that connects images to 3D geometry---stereo vision, structure from motion, visual SLAM, augmented reality---requires knowing the camera parameters. An uncalibrated camera is like a ruler with no markings: you can see things, but you cannot measure them.
How the topics connect: We begin with the pinhole camera model to understand the fundamental projection geometry. Then lens distortion models the imperfections of real lenses. Intrinsic parameters encode the camera's internal properties (focal length, sensor center), while extrinsic parameters encode its pose in the world. Zhang's method shows how to estimate intrinsics from a checkerboard. Stereo calibration extends this to camera pairs, and multi-camera systems generalize to arbitrary camera arrays.
Click any topic to jump in
The idealized projection from 3D to 2D — similar triangles, focal length, and perspective division.
Modeling the real camera
Real lenses bend straight lines — radial and tangential distortion models correct for this.
The camera matrix K — focal length in pixels, principal point, and skew encoded in one 3x3 matrix.
Where is the camera? [R|t] encodes position and orientation in the world.
Calibration methods
Estimate K from checkerboard images — homographies, constraints, and nonlinear refinement.
Calibrating camera pairs — the extrinsic relationship between two cameras.
Camera arrays, chain calibration, and synchronized capture for complex multi-view setups.
The pinhole camera is the simplest and most important camera model in computer vision. It describes how a 3D point in the world projects through a tiny aperture onto a flat image plane, forming a 2D image. Despite its simplicity, this model captures the essential geometry of perspective projection and serves as the foundation for all camera-related mathematics in 3D vision.
In the pinhole model, light from a 3D point passes through a single point (the optical center) and lands on the image plane at distance (the focal length) behind it. By similar triangles, a point in camera coordinates projects to image coordinates . This division by is called perspective division and is responsible for the foreshortening effect: distant objects appear smaller. The optical axis is the line through the pinhole perpendicular to the image plane, and the principal point is where this axis intersects the image plane.
A 3D point projects to on the image plane at distance . This dependence is the defining feature of perspective projection: it creates foreshortening (far objects shrink), vanishing points (parallel lines converge), and occlusion ordering. The projection is a many-to-one mapping --- all points along the ray for project to the same image point, which is why depth is lost in a single view and stereo or multi-view methods are needed to recover it.
A 1.8-meter-tall person stands 10 meters from a pinhole camera with focal length f = 50mm. How tall is their image on the sensor?
The focal length determines the camera's field of view (FOV). A short focal length produces a wide FOV (wide-angle lens), capturing more of the scene but with more perspective distortion. A long focal length produces a narrow FOV (telephoto lens), magnifying distant objects but capturing less of the scene. The relationship is . In the intrinsic matrix, focal length is expressed in pixels: .
The field of view relates the sensor width and focal length . At , the FOV is . Doubling halves the angular extent approximately (for small angles): radians. In pixel units, . A 35mm lens on a 36mm sensor gives FOV ; a 200mm lens gives FOV . The depth of field (range of distances appearing sharp) also scales with , which is why telephoto lenses have extremely shallow focus.
A camera has focal length f = 35mm and a sensor width of 36mm. What is the horizontal field of view?
The full pinhole projection equation combines intrinsic and extrinsic parameters into a single projection matrix . Given a 3D world point in homogeneous coordinates, the pixel location is found by: , then dividing by the third component. The intrinsic matrix is (focal lengths, principal point, skew), and is the extrinsic matrix (rotation and translation from world to camera). This compact equation is the mathematical backbone of multi-view geometry.
The full projection combines intrinsics () and extrinsics () into the projection matrix . This matrix has 11 DOF: 5 intrinsic (fx, fy, cx, cy, skew) + 6 extrinsic (3 rotation + 3 translation). The result is a homogeneous 3-vector; dividing by the third component yields pixel coordinates. This single matrix multiplication encodes the entire camera model, which is why decomposing back into , , (camera resectioning) is a fundamental operation.
A camera has , , . What pixel does the world point project to?