Face Alignment with Landmarks

hardPROFace RecognitionCVCV: Recognition

Implement a face alignment system using facial landmarks to normalize face images, enhancing recognition accuracy. This process involves computing a similarity transform to adjust the face image based on detected landmarks such as eyes, nose, and mouth.

Write align_face(image, landmarks, output_size), where output_size is (width, height) of the aligned crop.

The goal is to align the face image so that the eye centers are horizontally aligned, the inter-eye distance is fixed, and the image is cropped to a standard size, which can be achieved through a combination of translation, rotation, and scaling. These transformations can be represented using affine matrices, which describe linear transformations in 2D space.

To achieve this, the following steps are necessary:

Calculate the centers of the eyes from the detected landmarks. With the 68-point layout, the left eye is the mean of points 36–41 and the right eye the mean of points 42–47; with a short landmark list, row 0 is the left eye center and row 1 the right eye center. Landmarks are $(x, y)$ pixel coordinates.
Determine the angle of rotation required to align the eyes horizontally.
Compute the scale factor needed so that, after alignment, the inter-eye distance equals $\text{width} / 2.5$ pixels (e.g. 44.8 px for a 112-wide crop).
Apply the calculated transformations to the face image, mapping the midpoint between the eyes to the center of the crop, $(\text{width}/2, \text{height}/2)$ . Sample with bilinear interpolation; output pixels that map outside the source image are 0.

\begin{pmatrix} x' \\ y' \end{pmatrix} = \begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} + \begin{pmatrix} t_x \\ t_y \end{pmatrix}

The result has shape (height, width, 3) (taken from output_size, not from the input image) and the same uint8 dtype as the input.

This technique is widely used in face recognition systems to improve the accuracy of facial feature matching.

Example

Input

Face image and landmarks

Output

Aligned face

Reasoning

image is an $H \times W \times 3$ RGB array; landmarks holds $(x, y)$ pixel coordinates, rows 0 and 1 being the left and right eye; output_size is the crop's (width, height). Example: a $200 \times 200$ image, eyes at $(70, 90)$ and $(130, 90)$ , crop $112 \times 112$ .
Eye vector: $\Delta x = 60$ , $\Delta y = 0$ , so $\theta = \arctan(0/60) = 0$ — already level, no rotation. Distance $d = 60$ .
Scale: target inter-eye distance is $112 / 2.5 = 44.8$ px, so $s = 44.8 / 60 \approx 0.7467$ — the face shrinks.
Translation: the eye midpoint $(100, 90)$ moves to the crop centre $(56, 56)$ , so the eyes land at $(33.6, 56)$ and $(78.4, 56)$ .
Output: the Aligned face, a (112, 112, 3) uint8 array (height, width, channels); areas mapping outside the source are 0.
Notice: tilted eyes at $(70, 80)$ and $(130, 100)$ give $\theta = \arctan(20/60) \approx 18.43^\circ$ and $d \approx 63.25$ ; rotating by $-18.43^\circ$ with $s \approx 0.7084$ lands them on the same two output points.

Constraints

Input parameters: image (3D numpy array, RGB, shape=(height, width, 3)), landmarks (2D numpy array, shape=(n_landmarks, 2)) where n_landmarks is at least 4 (two eyes, nose, mouth)
Valid ranges: pixel values in [0, 255], landmark coordinates within image boundaries
Output format: aligned face image as 3D numpy array (RGB, shape=(height, width, 3)), uint8 precision
Special conditions: assume at least two eye landmarks are provided, and the inter-eye distance is fixed at 60 pixels after transformation
Assumptions: input image is not empty, and landmarks are detected with sufficient accuracy for alignment purposes

More from CV: Recognition

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results

Face Alignment with Landmarks

Example

Constraints

More from CV: Recognition

Background Knowledge

Algorithm/Approach

Continue the full explanation

Editor locked