Surface Integration from Normals

mediumPROPhotometric StereoCVCV: 3D Reconstruction

Implement a method to integrate surface normals and recover a depth map, a crucial step in 3D Reconstruction using Photometric Stereo. This process involves calculating the surface gradient from given normals.

The surface gradient is a fundamental concept in Computer Vision, representing the rate of change of the surface depth in the x and y directions. Given the surface normals $n_x$ , $n_y$ , $n_z$ , the surface gradient can be calculated as $p = -n_x/n_z$ and $q = -n_y/n_z$ , which are essential for integrating the surface.

To integrate the surface, follow these steps:

Calculate the surface gradient $p$ and $q$ from the given normals.
Integrate the gradient to obtain the depth $Z$ : accumulate $p$ along each row (left to right) and $q$ down each column (top to bottom), and add the two.
Shift the result so the mean depth is zero (absolute depth cannot be recovered from normals).

p = -n_x/n_z, \quad q = -n_y/n_z

Z_{ij} = \sum_{k=0}^{j} p_{ik} + \sum_{k=0}^{i} q_{kj}, \qquad Z \leftarrow Z - \bar{Z}

Return a depth map of shape (height, width).

This technique is widely used in 3D scanning and object recognition applications.

Example

Input

Normal map

Output

Depth map

Reasoning

Input: a $2 \times 3 \times 3$ normal map where every pixel holds the same unit normal $(n_x, n_y, n_z) = (-0.4472, 0, 0.8944)$ , i.e. a plane tilted only in $x$ .
Gradient: $p = -(-0.4472)/0.8944 = 0.5$ and $q = -0/0.8944 = 0$ at every pixel, so depth rises by $0.5$ per step to the right and does not change going down.
Integrate: accumulating $p$ along each row gives $0.5, 1.0, 1.5$ ; accumulating $q$ down each column adds $0$ , so both rows are $[0.5, 1.0, 1.5]$ .
Centre: the mean depth is $\bar{Z} = 1.0$ , and subtracting it gives $\begin{bmatrix} -0.5 & 0 & 0.5 \\ -0.5 & 0 & 0.5 \end{bmatrix}$ , one depth per pixel (shape $2 \times 3$ ).
Notice that normals only fix slopes, never absolute height, which is why the result is centred on zero. A flat surface (all normals $(0,0,1)$ , as in the $10 \times 10$ test) gives $p = q = 0$ and an all-zero depth map.

Constraints

Input normals: 2D numpy array of shape (height, width, 3) with dtype float32, representing the surface normals $n_x$ , $n_y$ , $n_z$
Valid ranges: $n_z$ values are non-zero, and all normal values are normalized to have a length of 1
Output depth map: 2D numpy array of shape (height, width) with dtype float32, representing the recovered depth $Z$ with precision up to 6 decimal places
Special conditions: The input normals are assumed to be noise-free and the surface is assumed to be Lambertian and convex
Boundary conditions: The depth values at the boundary of the image are assumed to be zero, and the Poisson equation solver should handle the boundary conditions accordingly

Learn this in

3D ReconstructionCV Study Plan

More from CV: 3D Reconstruction

NeRF Ray Samplinghard Marching Cubes SDF to Meshmedium ICP Point Cloud Alignmenthard NeRF Positional Encodingmedium Marching Cubes Configurationhard

Concept primerPREVIEW

Background Knowledge

Photometric Stereo is a technique used in Computer Vision to recover the 3D shape of an object from 2D images. It involves capturing images of the object under different lighting conditions and then using the variations in intensity to estimate the surface normals. The surface normals are essential in understanding the orientation of the surface at each point. In this problem, we are given the surface normals (nx, ny, nz) and need to integrate them to recover the depth map.

The surface gradient is a fundamental concept in this problem, which is represented by the equations $p = -n_x/n_z$ and $q = -n_y/n_z$ . These equations relate the surface normals to the partial derivatives of the depth map (Z) with respect to the x and y coordinates. The Poisson equation is a partial differential equation that describes the relationship between the surface gradient and the depth map. It is a powerful tool for solving surface integration problems.

In the context of 3D Reconstruction, integrating surface normals to recover depth maps is a crucial step. The Poisson equation solver is a numerical method used to solve the Poisson equation and obtain the depth map. It is an essential component of many 3D reconstruction algorithms, including Photometric Stereo. Understanding the underlying mathematics and numerical methods is vital for successfully solving this problem.

Algorithm/Approach

The general approach to solving this problem involves using a Poisson equation solver to integrate the surface gradient and recover the depth map. The algorithm pattern typically includes the following steps: (1) compute the surface gradient from the given normals, (2) setup the Poisson equation using the surface gradient, and (3) solve the Poisson equation using a numerical method.

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.