Ordinal Depth Ranking Loss: Background & Implementation Guide

Background Knowledge

Depth Estimation and Relative Ordering: In computer vision, monocular depth estimation predicts spatial depth from a single 2D image. While absolute depth values are difficult to predict accurately from monocular images, the relative ordering of depths—which pixels are closer or farther—is often more reliable and easier to learn. This is the core insight behind ordinal depth loss: instead of penalizing absolute depth errors, we penalize violations of the ground truth depth ordering between pixel pairs.

Ordinal Regression in Computer Vision: Ordinal regression treats depth prediction as a ranking problem rather than a pure regression problem. The key difference is that ordinal methods exploit the natural ordering structure in the data (closer < farther) to improve learning efficiency and generalization. This approach has proven effective in depth estimation because it aligns with how humans perceive depth—we naturally judge relative distances before estimating absolute values.

Why Margin-Based Ranking Loss?: The margin parameter $\tau$ allows the loss to tolerate small ordering violations without penalty. This is crucial because: (1) it prevents the network from over-fitting to noisy ground truth labels, (2) it provides a "safety zone" where predictions don't need to be perfect, and (3) it encourages the network to focus on clear ordering violations rather than minor depth fluctuations.

Algorithm/Approach

The ordinal depth ranking loss operates on pairwise comparisons of pixels:

1. Identify the ground truth ordering: For each pair of pixels $(i, j)$, determine which is closer using $\text{sign}(Z_i - Z_j)$
2. Check predicted ordering: Compute the predicted depth difference $($\hat{Z}$_i -$\hat{Z}$_j)$
3. Measure violation: Multiply the sign of ground truth ordering by the predicted difference to detect violations
4. Apply margin: Only penalize violations that exceed the margin threshold $\tau$
5. Aggregate: Sum losses across all pixel pairs (or a sampled subset for efficiency)

This is a ranking-based loss function, similar to approaches used in object detection and metric learning.

Step-by-Step Implementation Strategy

Step 1: Understand the Loss Formula

Break down the formula component by component:

• $\text{sign}(Z_i - Z_j)$ produces: $+1$ if $Z_i > Z_j$, $-1$ if $Z_i < Z_j$, $0$ if equal
• $($\hat{Z}$_i -$\hat{Z}$_j)$ is the predicted depth difference
• The product $\text{sign}(Z_i - Z_j) \cdot ($\hat{Z}$_i -$\hat{Z}$_j)$ is positive when predictions agree with ground truth, negative when they disagree
• Negating this product: $-$\text{sign}$(Z_i - Z_j) \cdot ($\hat{Z}$_i -$\hat{Z}$_j)$ becomes negative for correct orderings and positive for violations
• $\max(0, \cdot + \tau)$ applies the margin: only penalize when violation exceeds $\tau$