Trajectory Smoothing Filter

1. Background Knowledge

Video stabilization tries to remove unwanted high‑frequency camera shake while preserving the intended low‑frequency camera motion (pans, tilts, zooms). The camera motion between two frames is often modeled as a 2D transform: translation, rotation, scale, or a full homography $H \in \mathbb{R}^{3\times 3}$. By chaining these inter‑frame transforms, you get a camera trajectory over time (e.g., trajectory of camera pose parameters or homographies).

This trajectory is typically noisy: hand shake appears as rapid, small oscillations in the parameter curves over time. Stabilization works by smoothing these parameter trajectories with a temporal filter (moving average, Gaussian, low‑pass, etc.). The smoothed trajectory T_{\text{smooth}}(t) describes how the virtual stabilized camera should move. You then compute a per‑frame correction transform $C(t)$ that maps the original view at time $t$ to where it would be if the camera had followed the smoothed path instead of the original shaky one.

2. Algorithm / Approach Pattern

General pattern for trajectory‑based stabilization:

1. Estimate inter‑frame motion: For each pair of consecutive frames, estimate a 2D transform $M_t$ (e.g., homography) from frame $t$ to $t+1$.
2. Accumulate to get trajectory: Compose the transforms to obtain a global camera transform $T(t)$ for each frame (relative to a reference frame).
3. Smooth the trajectory: Apply a temporal filter to the sequence $\{T(t)\}$ (or to its parameter vectors) to get \{T_{\text{smooth}}(t)\}.
4. Compute correction transforms: For each frame, compute C(t) = T_{\text{smooth}}(t) \cdot T(t)^{-1}.
5. Warp frames: Apply $C(t)$ to frame $t$ to render the stabilized video.

Your problem focuses on steps 3–4: given $T(t)$, implement the smoothing and compute $C(t)$.

3. Step‑by‑Step Strategy

Assume your input is a list/array of transforms $T, T, \dots, T[N-1]$ (each could be a 3×3 homography or a parameter vector like $(dx, dy, \theta, s)$).

Step 1: Choose representation to smooth

• Common and simpler: convert each transform $T(t)$ to a parameter vector:
• For example, from a 3×3 homography restricted to similarity/affine or from a known model:
• translation: $(dx_t, dy_t)$
• rotation: $\theta_t$
• scale: $s_t$
• Then you get 1D time series for each parameter: $\{dx_t\}, \{dy_t\}, \{\theta_t\}, \{s_t\}$.

This avoids problems with directly averaging matrices.

Step 2: Design the smoothing filter

Given your formula

you are conceptually doing a finite impulse response (FIR) filter with weights $w_i$ over a temporal window.

For parameters:

# Example: uniform moving average with window size 2k+1
w = [1.0 / (2*k + 1)] * (2*k + 1) # or Gaussian weights normalized to sum to 1

For each frame index t, you’ll combine values from t-k to t+k with weights w[i].

Step 3: Apply smoothing per parameter

For each time $t$:

• For each parameter $p \in \{dx, dy, \theta, s\}$:

with boundary handling when $t+i$ is outside $[0, N-1]$ (e.g., clamp indices or shrink window).

In code (sketch):