PIXELBANKv8.2.1
Menu
Sign In

Depthwise Separable Convolution

MediumCNNs

Implement depthwise separable convolution, a key efficiency technique in MobileNets.

Standard convolution: Cin×H×W→Cout×H′×W′C_{in} \times H \times W \rightarrow C_{out} \times H' \times W' Parameters: Cout×Cin×K×KC_{out} \times C_{in} \times K \times K

Depthwise separable breaks this into:

  1. Depthwise: Each input channel convolved separately
    • Parameters: Cin×K×KC_{in} \times K \times K
  2. Pointwise: 1×1 convolution to mix channels
    • Parameters: Cout×CinC_{out} \times C_{in}

Efficiency gain: 1Cout+1K2\frac{1}{C_{out}} + \frac{1}{K^2} of standard conv!

Example:

Input:
input: (1, 3, 8, 8)  # RGB image
depthwise: (3, 1, 3, 3)  # 3x3 per channel
pointwise: (16, 3, 1, 1)  # Expand to 16 channels
Output:
tensor of shape (1, 16, 6, 6)
Reasoning:

Depthwise (no padding): (1,3,8,8) → (1,3,6,6) Each channel filtered independently

Pointwise: (1,3,6,6) → (1,16,6,6) Linear combination of channels at each spatial location

Total params: 3×9 + 16×3 = 27 + 48 = 75 Standard 3×3 conv: 16×3×9 = 432 (5.8× more!)

Constraints:

  • input: Tensor (batch, channels, height, width)
  • depthwise_kernel: (channels, 1, k, k)
  • pointwise_kernel: (out_channels, channels, 1, 1)
  • Return: Output tensor
Editor

Test Results

0/0
Run code to see test results.