Gradient Descent for Linear Regression

Implement batch gradient descent for simple linear regression.

Given data points $(x, y)$ , initial weight $w$ and bias $b$ , learning rate $\alpha$ , and number of iterations, update $w$ and $b$ to minimize Mean Squared Error:

$MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - (w \cdot x_i + b))^2$

The gradients are: $\frac{\partial MSE}{\partial w} = \frac{-2}{n} \sum_{i=1}^{n} x_i(y_i - (w \cdot x_i + b))$ $\frac{\partial MSE}{\partial b} = \frac{-2}{n} \sum_{i=1}^{n} (y_i - (w \cdot x_i + b))$

Return a tuple (w, b) after all iterations, both rounded to 4 decimal places.

Example:

Input:

X = [1, 2, 3]
y = [2, 4, 6]
w = 0.0, b = 0.0
learning_rate = 0.1, iterations = 100

Output:

(1.9715, 0.0647)

Reasoning:

We start with the given data points $(x, y)$ : $(1, 2)$ , $(2, 4)$ , $(3, 6)$ , initial weight $w = 0.0$ , bias $b = 0.0$ , learning rate $\alpha = 0.1$ , and number of iterations $= 100$ .
We calculate the gradients $\frac{\partial MSE}{\partial w}$ and $\frac{\partial MSE}{\partial b}$ using the given formulas: $\frac{\partial MSE}{\partial w} = \frac{-2}{n} \sum_{i=1}^{n} x_i(y_i - (w \cdot x_i + b))$ and $\frac{\partial MSE}{\partial b} = \frac{-2}{n} \sum_{i=1}^{n} (y_i - (w \cdot x_i + b))$ , and update $w$ and $b$ using $w = w - \alpha \cdot \frac{\partial MSE}{\partial w}$ and $b = b - \alpha \cdot \frac{\partial MSE}{\partial b}$ for each iteration.
We repeat the process of calculating gradients and updating $w$ and $b$ for $100$ iterations, which minimizes the Mean Squared Error $MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - (w \cdot x_i + b))^2$ .
After $100$ iterations, we obtain the updated values of $w$ and $b$ , which are then rounded to $4$ decimal places, resulting in the output $(1.9715, 0.0647)$ .

Constraints:

X and y are lists of equal length
learning_rate > 0, iterations >= 1
Initial w and b are given as floats
Return tuple (w, b) rounded to 4 decimal places

Example:

Constraints:

Test Results