Weighted HDR Merge: Background Knowledge & Implementation Guide

Background Knowledge

HDR Imaging Fundamentals

High Dynamic Range (HDR) imaging addresses a fundamental limitation of digital cameras: a single exposure cannot simultaneously capture detail in both bright and dark regions of a scene. A camera's sensor has a limited dynamic range—the ratio between the brightest and darkest values it can distinguish. When you expose for highlights, shadows become underexposed and noisy; when you expose for shadows, highlights saturate to white. HDR merging solves this by combining multiple exposures taken at different shutter speeds, each optimized for different brightness regions. Short exposures preserve highlight detail without saturation, while long exposures capture shadow detail with better signal-to-noise ratios.

Linearization and Radiometric Calibration

Camera sensors apply gamma correction (typically γ = 2.2) to compress the sensor's linear response into an 8-bit or 16-bit integer range suitable for storage. This gamma curve is nonlinear: pixel values don't correspond directly to physical light intensity. To merge exposures meaningfully, you must first linearize each exposure by inverting this gamma correction, converting pixel values back to linear radiance proportional to actual light energy. The exposure time $t_i$ acts as a scaling factor—longer exposures accumulate more photons, so the same scene brightness produces higher pixel values. By dividing by exposure time, you normalize for this effect and recover comparable radiance estimates across different exposures.

Weighted Averaging and Quality Assessment

Not all pixels in all exposures are equally reliable. A pixel in a short exposure that's near saturation (value close to 255) carries high uncertainty about the true brightness—it's clipped. Similarly, a pixel in a long exposure that's very dark carries high noise. Weighting schemes assign higher confidence to pixels in the "middle" of the exposure range where the sensor operates most linearly and with best signal-to-noise. The weighted average formula normalizes by the sum of weights, ensuring the result is independent of the absolute weight magnitudes.

Algorithm/Approach

The general approach follows these stages:

1. Input Processing: Load multiple exposures and their corresponding exposure times.
2. Linearization: Convert each exposure from gamma-corrected pixel space to linear radiance space using the inverse gamma function and exposure time normalization.
3. Weight Calculation: Compute per-pixel weights based on exposure quality (typically favoring mid-range pixel values and penalizing near-black or near-saturated pixels).
4. Weighted Fusion: Apply the weighted averaging formula to combine linearized radiances.
5. Output Generation: Convert the merged HDR radiance back to a displayable format (tone mapping or log encoding).

This is a pixel-wise operation—each output pixel is computed independently from corresponding pixels across all input exposures.

Step-by-Step Strategy

Step 1: Linearize Each Exposure

• For each exposure $i$ and each pixel, apply: L_i = \frac{(Z_i / 255)^\gamma}{t_i}
• Normalize pixel values to [0, 1] by dividing by 255
• Apply the inverse gamma curve (raise to power $\gamma$)
• Divide by exposure time to account for integration duration
• Result: linear radiance in physical units

Step 2: Design a Weight Function

• Create a weighting function $w(Z)$ that maps pixel values to confidence scores
• Typically, this is a bell curve or triangular function peaking at mid-range values (e.g., around 127–128 for 8-bit images)
• Penalize very dark pixels (high noise) and very bright pixels (saturation/clipping)
• Common choice: w(Z) = \begin{cases} Z/127.5 & \text{if }Z \leq 127.5 \\ (255-Z)/127.5 &\text{if } Z > 127.5 \end{cases} (tent function)

Step 3: Compute Per-Pixel Weights

• For each pixel position and each exposure, evaluate $w_i = w(Z_i)$
• Handle edge cases: if all weights are zero (rare), use uniform weights or skip the pixel

Step 4: Apply Weighted Averaging

• For each pixel, compute: H = \frac{\sum_i w_i \cdot L_i}{\sum_i w_i}
• Accumulate weighted radiances in the numerator
• Accumulate weights in the denominator
• Divide to get the final merged HDR value

Step 5: Handle Output

• Store the result as floating-point HDR data (e.g., OpenEXR format)
• Optionally apply tone mapping for visualization on standard displays

Common Pitfalls

• Forgetting to linearize: Working directly with gamma-corrected pixel values produces incorrect results because the averaging happens in the wrong color space.
• Ignoring exposure time: Failing to normalize by $t_i$ causes long exposures to dominate the average artificially.
• Poor weight design: Using uniform weights or weights that don't penalize saturation/noise defeats the purpose of multi-exposure fusion. Test your weight function visually.
• Division by zero: If all weights are zero at a pixel (unlikely but possible with extreme weight functions), handle gracefully—use a fallback or skip.
• Numerical precision: Radiance values can span many orders of magnitude. Use double precision or carefully manage floating-point ranges to avoid underflow/overflow.
• Misalignment: This algorithm assumes exposures are perfectly registered (aligned). Real-world images may have slight motion; consider alignment preprocessing if needed.
• Gamma value assumption: The problem states γ = 2.2, but verify this for your camera model; some devices use different curves.

Time & Space Complexity

Time Complexity: $O(N \cdot M \cdot K)$

• $N$ = number of pixels (image width × height)
• $M$ = number of exposures
• $K$ = constant operations per pixel per exposure (linearization, weighting, accumulation)
• Each pixel is processed independently across all exposures; no nested loops over spatial neighborhoods.

Space Complexity: $O(N \cdot M)$

• Store all input exposures: $O(N \cdot M)$
• Output HDR image: $O(N)$
• Temporary accumulators (numerator, denominator): $O(N)$
• Overall dominated by input storage; can be optimized to $O(N)$ by processing exposures sequentially if memory is tight.