MSE Loss Gradient

easyPROBackpropagationDeep LearningCV: Deep Learning

Implement a function to compute the gradient of the Mean Squared Error (MSE) loss with respect to predictions. The MSE loss is a common loss function used in regression problems to measure the difference between predicted and actual values.

The MSE loss is calculated as the average of the squared differences between predicted and actual values, which can be expressed as $MSE = \frac{1}{n}\sum_{i=1}^{n}(\hat{y}_i - y_i)^2$ . To update model parameters using backpropagation, we need to compute the gradient of the MSE loss with respect to each prediction.

Here are the steps to compute the gradient:

Calculate the difference between each predicted value and actual value.
Square each difference and sum them up.
Compute the average of the squared differences.
Calculate the gradient of the MSE loss with respect to each prediction.

\frac{\partial MSE}{\partial \hat{y}_i} = \frac{2}{n}(\hat{y}_i - y_i)

This technique is widely used in deep learning models for regression tasks.

Example

Input

mse_grad([2, 4], [1, 3])

Output

[1.0, 1.0]

Reasoning

Inputs: pred holds the model's predictions $\hat{y} = [2, 4]$ and target holds the true values $y = [1, 3]$ , so there are $n = 2$ samples.
Errors: each prediction overshoots its target by the same amount: $\hat{y}_1 - y_1 = 2 - 1 = 1$ and $\hat{y}_2 - y_2 = 4 - 3 = 1$ . For reference the loss itself is $\frac{1}{2}(1^2 + 1^2) = 1$ , but the gradient doesn't need it.
Gradient: plug each error into $\frac{\partial MSE}{\partial \hat{y}_i} = \frac{2}{n}(\hat{y}_i - y_i)$ . The factor is $\frac{2}{2} = 1$ , giving $1 \cdot 1 = 1.0$ for both entries, so the output is [1.0, 1.0].
Output: one gradient value per prediction, same length as the inputs, rounded to 4 decimal places.
Notice: the sign tells the direction. Both values are positive because both predictions are too high, so lowering them reduces the loss. A prediction below its target (e.g. $\hat{y} = 0$ , $y = 1$ , $n = 1$ ) gives $2 \cdot (0 - 1) = -2.0$ , and a perfect prediction gives $0.0$ .

Constraints

Return gradient rounded to 4 decimal places

Learn this in

Deep LearningCV Study Plan

More from CV: Deep Learning

Implement Max Pooling with Indiceshard Depthwise Separable Convolutionmedium Softmax Cross-Entropy Gradientmedium Batch Normalization Forward Passhard Flatten Feature Mapeasy

Concept primerPREVIEW

You are given the gradient formula already; your task is to understand where it comes from and how to apply it correctly in code.

1. Background Knowledge

Mean Squared Error (MSE) compares predictions $\hat{y}$ to targets $y$ by averaging squared differences:

MSE = \frac{1}{n} \sum_{i=1}^n (\hat{y}_i - y_i)^2 It penalizes larger errors more heavily because of the square. - **Gradients for backpropagation**: during training, we update model parameters by moving them in the direction that *reduces* the loss. This uses the gradient of the loss with respect to: 1. The model outputs (predictions) \hat{y}_i 2. Then, via chain rule, with respect to weights and biases inside the network. - **Chain rule idea**: if a parameter w affects the loss through \hat{y}, then \frac{\partial MSE}{\partial w} = \sum_i \frac{\partial MSE}{\partial \hat{y}_i} \cdot \frac{\partial \hat{y}_i}{\partial w} So the derivative \frac{\partial MSE}{\partial \hat{y}_i} is the “entry point” for backprop. --- ## 2. Algorithm / General Approach For MSE loss: 1. **Define the loss** as average of squared errors: (\hat{y} - y)^2 averaged over all samples (and sometimes over all output dimensions). 2. **Differentiate w.r.t. each prediction** \hat{y}_i using basic calculus (derivative of x^2 is 2x). 3. **Use this gradient** as the “error signal” at the output layer, then propagate it backward through the network using the chain rule. In array/vector form this becomes:

$\nabla$ _{ $\hat{y}$ } $MSE =$ \frac{2}{n} $(\hat{y}$ - y)

--- ## 3. Step-by-Step Strategy To derive and then implement: 1. **Write the loss explicitly**:

$MSE =$ \frac{1}{n}\sum_{i=1} $^n (\hat{y}$ _i - y_i)^2

2. **Focus on a single term** in the sum for index i:

$L_i =$ \frac{1}{n} $(\hat{y}$ _i - y_i)^2

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.