Depthwise Separable Convolution

Implement depthwise separable convolution, a key efficiency technique in MobileNets.

Standard convolution: $C_{in} \times H \times W \rightarrow C_{out} \times H' \times W'$ Parameters: $C_{out} \times C_{in} \times K \times K$

Depthwise separable breaks this into:

1. Depthwise: Each input channel convolved separately
   • Parameters: $C_{in} \times K \times K$
2. Pointwise: 1×1 convolution to mix channels
   • Parameters: $C_{out} \times C_{in}$

Efficiency gain: $\frac{1}{C_{out}} + \frac{1}{K^2}$ of standard conv!