1. Background Knowledge
PyTorch nn.Module is the foundation for all neural networks. Every neural network in PyTorch inherits from nn.Module, which provides essential functionality like parameter tracking, device placement, and the forward pass mechanism.
Key concepts:
- Parameters: nn.Module automatically registers layers like nn.Linear as trainable parameters via self.layer = nn.Linear(...)
- Forward pass: Defined in forward() method; called automatically when using model(input)
- Linear layer: nn.Linear(in_features, out_features) implements y=xWT+b where W∈Rout×in, b∈Rout
- Shape transformation: Input (batch_size, input_size) → Output (batch_size, output_size)
Prerequisites:
import torch
import torch.nn as nn
2. Algorithm Approach
This is a feedforward neural network with a single linear transformation:
Input: (batch_size, 10) → Linear(10, 5) → Output: (batch_size, 5)
Mathematical operation: \mathbf{y}=\mathbf{x}\mathbf{W}T+\mathbf{b}
- \mathbf{x}∈Rbatch×10
- \mathbf{W}∈R5×10 (weights)
- \mathbf{b}∈R5 (bias)
- \mathbf{y}∈Rbatch×5
3. Step-by-Step Strategy
- Define the class inheriting from nn.Module:
class SimpleNet(nn.Module):
- Initialize in init():
- Call super(SimpleNet, self).init() first
- Create self.linear = nn.Linear(10, 5)
- Define forward():
- Return self.linear(x) (no other operations needed)
- Create create_simple_network():
- Instantiate SimpleNet()
- Verify it's nn.Module subclass: isinstance(net, nn.Module)
- Test output shape: net(torch.randn(1, 10)).shape
- Return dictionary with all 3 keys
Complete solution structure:
def create_simple_network():
class SimpleNet(nn.Module): # Defined inside function
def __init__(self):
super().__init__()
self.linear = nn.Linear(10, 5)
def forward(self, x):
return self.linear(x)
net = SimpleNet()
return {
"class_name": "SimpleNet",
"is_module": isinstance(net, nn.Module),
"output_shape": list(net(torch.randn(1, 10)).shape)
}
4. Common Pitfalls
| Mistake | Why it fails | Fix |
|---|
| Forget super().init() | Parameters not registered | Always call first in init |
| Call net.forward(x) | Bypasses hooks/autocast | Use net(x) |
| Wrong shape test | Input must be (1, 10) | torch.randn(1, 10) |
| Return class instead of dict | Test expects dictionary | Return {"class_name":...,...} |
| nn.Linear(5, 10) | Wrong dimensions | nn.Linear(10, 5) per spec |
Critical: Class must be named exactly "SimpleNet" for the test.
5. Time & Space Complexity
Forward pass: O(n⋅din⋅dout)=O(1⋅10⋅5)=O(50) per sample
- Matrix multiplication dominates
- Constant time for fixed architecture
Space complexity:
- Parameters: 10×5+5=55 scalars
- Activations: O(batch_size⋅5)
- Total: O(1) (fixed-size model)
Verification test runs in O(1) time since batch_size=1 and layers are tiny.
This pattern scales to complex networks: more layers → O(\sumLi⋅di⋅di+1).