Chain Rule Application

mediumPROBackpropagationDeep LearningCV: Deep Learning

Implement a function to apply the chain rule in the context of backpropagation for computing gradients in deep learning models. This task involves calculating the gradient of a loss function $L$ with respect to an input $x$ , given the upstream gradient $\frac{dL}{dy}$ and the local gradient $\frac{dy}{dx}$ .

The chain rule is a fundamental concept in calculus that allows us to compute the derivative of a composite function. In the context of neural networks, it is crucial for backpropagation, where we need to propagate error gradients through multiple layers to update model parameters. The chain rule states that the derivative of a composite function $L(y(x))$ with respect to $x$ can be expressed as the product of the derivatives of $L$ with respect to $y$ and $y$ with respect to $x$ , i.e., $\frac{dL}{dx} = \frac{dL}{dy} \cdot \frac{dy}{dx}$ .

Here upstream and local are lists of the same length, one entry per component of $x$ , and the product is taken component by component: $\left(\frac{dL}{dx}\right)_i = \left(\frac{dL}{dy}\right)_i \cdot \left(\frac{dy}{dx}\right)_i$ . Return the result as a list of the same length.

To compute the gradient $\frac{dL}{dx}$ , we can follow these steps:

Receive the upstream gradient $\frac{dL}{dy}$ from the previous layer.
Compute the local gradient $\frac{dy}{dx}$ using the activation functions and weights of the current layer.
Apply the chain rule by multiplying the upstream gradient with the local gradient.

\frac{dL}{dx} = \frac{dL}{dy} \cdot \frac{dy}{dx}

This technique is widely used in training deep neural networks.

Example

Input

chain_rule([0.5, 0.5], [1, 0])

Output

[0.5, 0.0]

Reasoning

upstream holds $\frac{dL}{dy}$ : how much the loss changes per unit change in each component of $y$ . Here both components have sensitivity $0.5$ .
local holds $\frac{dy}{dx}$ for each component: $y_1$ moves one-for-one with $x_1$ (slope $1$ ), while $y_2$ does not respond to $x_2$ at all (slope $0$ ), like a ReLU unit whose input was negative.
The chain rule pairs each component with its own slope: $\frac{dL}{dx_1} = 0.5 \times 1 = 0.5$ and $\frac{dL}{dx_2} = 0.5 \times 0 = 0$ .
Each value is rounded to 4 decimal places, giving [0.5, 0.0], a gradient with one entry per input component.
Notice that a zero local gradient blocks the signal completely: however large the upstream gradient is, nothing reaches $x_2$ . This is how gradients "die" behind inactive units during backpropagation.

Constraints

Return gradient rounded to 4 decimal places

Learn this in

Deep LearningCV Study Plan Mathematical FoundationsFoundations Neural NetworksML Study Plan

More from CV: Deep Learning

Implement Max Pooling with Indiceshard Depthwise Separable Convolutionmedium Softmax Cross-Entropy Gradientmedium Batch Normalization Forward Passhard Flatten Feature Mapeasy

Chain Rule Application

Example

Constraints

Learn this in

More from CV: Deep Learning

1. Background Knowledge

2. Algorithm / General Approach

3. Step‑by‑Step Strategy

Continue the full explanation

Editor locked