Disparity to Depth Conversion: Background & Strategy

Background Knowledge

Stereo Vision Fundamentals

Stereo matching is a core technique in computer vision for estimating 3D depth from two or more 2D images. The fundamental principle mimics human binocular vision: by comparing how the same scene point appears in two cameras positioned at different locations, we can infer its distance. In a rectified stereo setup, the two camera images are geometrically aligned so that corresponding points lie on the same horizontal line, simplifying the matching problem to a 1D search along each scanline.

The Disparity-Depth Relationship

Disparity is the horizontal pixel offset between corresponding points in the left and right images. Objects closer to the camera appear more shifted between the two views (high disparity), while distant objects appear nearly identical (low disparity). This relationship is governed by the stereo geometry equation you've been given: the depth $Z$ is inversely proportional to disparity $d$. The proportionality constant $f \cdot B$ (focal length times baseline) encodes the camera calibration parameters. This inverse relationship is crucial: small errors in disparity estimation for distant objects translate to large depth errors, since depth changes rapidly with small disparity changes when $d$ is small.

Practical Considerations

In real stereo systems, several challenges arise: occluded regions (visible in one image but not the other), textureless areas (where matching is ambiguous), and invalid disparities (where no match is found). These typically result in missing or unreliable disparity values that must be handled carefully during conversion to avoid invalid depth estimates.

Algorithm/Approach

The solution follows a straightforward three-step pattern:

1. Input validation: Check for invalid disparity values (zero, negative, or unmatched pixels)
2. Direct conversion: Apply the stereo geometry formula to valid disparities
3. Output handling: Return depth values, handling edge cases appropriately

This is a direct mathematical transformation with no iterative or complex algorithmic components—the challenge lies in careful handling of edge cases and understanding the geometric principles.

Step-by-Step Strategy

Step 1: Understand Your Inputs

• Identify the disparity map (typically a 2D array where each pixel contains a disparity value)
• Obtain the calibration parameters: focal length $f$ (in pixels) and baseline $B$ (in world units, often meters)
• Understand the data type and range of disparity values in your input

Step 2: Handle Invalid Disparities

• Identify which disparity values are invalid (commonly: $d = 0$, negative values, or special sentinel values like NaN)
• Decide how to represent invalid depth: use NaN, a large sentinel value, or a mask array
• This prevents division by zero and marks regions where depth cannot be reliably estimated

Step 3: Apply the Conversion Formula

• For each valid disparity value, compute: $Z = \frac{f \cdot B}{d}$
• Ensure you're using the correct data types (floating-point for accurate results)
• Consider whether to pre-compute $f \cdot B$ to avoid redundant multiplication

Step 4: Validate Your Output

• Check that depth values are physically reasonable (positive, within expected range)
• Verify that the inverse relationship holds: smaller disparities produce larger depths
• Test edge cases: very small disparities (should give large depths), very large disparities (should give small depths)

Common Pitfalls

• Division by zero: Forgetting to check for $d = 0$ before division. Always validate disparity values first.
• Unit mismatches: Confusing pixel units with world units. Focal length must be in pixels; baseline in world units (e.g., meters). The result will be in the same units as baseline.
• Data type issues: Using integer division instead of floating-point, which loses precision. Ensure $f$, $B$, and $d$ are floats.
• Ignoring invalid regions: Not handling occluded or unmatched pixels properly, leading to NaN or inf values propagating through your output.
• Forgetting the inverse relationship: Misremembering the formula as $Z = \frac{d}{f \cdot B}$ instead of $Z = \frac{f \cdot B}{d}$.
• Numerical instability: Very small disparity values can cause numerical issues; consider adding a small epsilon or filtering out unreliable disparities.

Time & Space Complexity

• Time Complexity: $O(n)$ where $n$ is the total number of pixels in the disparity map. Each pixel requires one division operation (constant time).
• Space Complexity: $O(n)$ for storing the output depth map. If you're processing in-place or streaming, you could reduce this, but typically you'll need to store the full result.

The simplicity of this problem makes it an excellent introduction to stereo vision—the real complexity in practical systems comes from computing the disparity map accurately, not from converting it to depth.