📘

Per-Layer Learning Rates

Problem Statement

Configure different learning rates for different layers using parameter groups.

Background

PyTorch optimizers accept a list of parameter groups, each with its own hyperparameters. This is essential for fine-tuning where you want lower LR for pretrained layers.

Your Task

The starter code creates a 2-layer model. Create an Adam optimizer with different learning rates per layer: 0.001 for fc1 and 0.01 for fc2.

Output Format

Returns a dictionary with "num_param_groups", "group0_lr", "group1_lr", "group0_num_params", and "group1_num_params".

Example:

Input:

None

Output:

{'num_param_groups': 2, 'group0_lr': 0.001, 'group1_lr': 0.01, 'group0_num_params': 2, 'group1_num_params': 2}

Reasoning:

The function per_layer_lr_test() seeds the random number generator with torch.manual_seed(42) to ensure reproducibility.
A model is created with two linear layers (fc1 and fc2), and an Adam optimizer is initialized with two parameter groups: one for fc1 with a learning rate of $0.001$ , and one for fc2 with a learning rate of $0.01$ .
Each linear layer has two parameter tensors (weight and bias), so the number of parameter tensors in each group is $2$ .
The function returns a dictionary containing the number of parameter groups ( $2$ ), the learning rates of each group ( $0.001$ and $0.01$ ), and the number of parameter tensors in each group ( $2$ and $2$ ).

Constraints:

Two parameter groups with different LRs
fc1: lr=0.001, fc2: lr=0.01
Use Adam optimizer

Editor

Python 3.13.1

Test Results

0/0

Run code to see test results.

📘