FPN Feature Fusion (Simplified)

hardPROConvNextImage Processing, Neural NetworksAdvanced Architectures

Implement a Feature Pyramid Network (FPN) fusion module, a crucial component in ConvNext architectures for multi-scale object detection. This module combines low-resolution semantic features with high-resolution spatial features.

The Feature Pyramid Network (FPN) is a technique used to leverage the benefits of both high-resolution and low-resolution feature maps. In a typical ConvNext architecture, feature maps $C_3, C_4, C_5$ are generated by the backbone, where $C_5$ is the deepest and smallest map. To fuse these features, the FPN uses a top-down pathway with upsampling and element-wise addition.

Here are the steps to compute $P_4$ :

Upsample $C_5$ by 2× using nearest-neighbor interpolation
Element-wise add upsampled $C_5$ to $C_4$

P_4 = \text{Upsample}(C_5, 2\times) + C_4

This technique is widely used in object detection tasks, such as those found in autonomous vehicles.

Example

Input

C4 (8×8, all 2s), C5 (4×4, all 4s)

Output

8×8 matrix, all values 6.0 (upsampled 4 + original 2)

Reasoning

Each 1×1 cell in C5 becomes a 2×2 block with value 4. Adding to C4 (value 2) gives 6.

Constraints

$C_3$ dimensions: $H \times W$ (e.g., 16×16)
$C_4$ dimensions: $H/2 \times W/2$ (e.g., 8×8)
$C_5$ dimensions: $H/4 \times W/4$ (e.g., 4×4)
All values are scalar intensities

More from Advanced Architectures

Complete Mixer Block with Residualshard Channel Mixing with GELUmedium Token Mixing Layermedium Bidirectional Scan for Visionmedium Discretize Continuous State Space Modelhard

Concept primerPREVIEW

1. Background Knowledge

Feature Pyramid Networks (FPN) address multi-scale object detection by fusing high-resolution spatial details from shallow layers with low-resolution semantic richness from deep layers. Backbone networks (e.g., ResNet) produce feature maps $C_3, C_4, C_5$ where depth increases semantic content but reduces spatial resolution: $C_3$ (e.g., $16 \times 16$ ), $C_4$ ( $8 \times 8$ ), $C_5$ ( $4 \times 4$ ). The top-down pathway in FPN uses nearest-neighbor upsampling (×2) and element-wise addition (lateral connections) to create pyramid levels like $P_4$ , balancing scale variance without heavy computation.

Prerequisites: Tensor operations (upsampling, addition), CNN feature maps as $(H, W, C)$ tensors, nearest-neighbor interpolation preserves values by replicating pixels.

2. Algorithm Approach

Standard FPN uses a top-down fusion:

Upsample deepest map ( $C_5 \to 8 \times 8$ ) via nearest-neighbor (scale=2).
Add to same-resolution lateral map ( $C_4$ ): $P_4 =$ \text{Upsample} $(C_5) + C_4$ .

In PyTorch/TensorFlow:

import torch.nn.functional as F

# Upsample C5 to C4 size
upsampled_c5 = F.interpolate(C5, scale_factor=2, mode='nearest')
P4 = upsampled_c5 + C4 # Assumes matching channels/spatial dims

Variants (e.g., BiFPN, NAS-FPN) add bottom-up paths or attention, but this task is pure top-down.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results