📘

Multiple Linear Regression (Normal Equation)

Implement multiple linear regression using the normal equation.

Given a feature matrix $X$ (without bias column) and target vector $y$ , compute the weight vector $w$ that minimizes the mean squared error.

First, prepend a column of ones to $X$ to form the augmented matrix $X_a$ , then solve: $w = (X_a^T X_a)^{-1} X_a^T y$

Return the weight vector as a list, rounded to 4 decimal places. The first element is the bias/intercept.

You must implement matrix operations from scratch (transpose, multiply, invert).

Input:

X = [[1], [2], [3]]
y = [2, 4, 6]

Output:

[0.0, 2.0]

Reasoning:

First, we prepend a column of ones to $X$ to form the augmented matrix $X_a$ : $X_a = [[1, 1], [1, 2], [1, 3]]$
Then, we calculate $X_a^T X_a$ and $X_a^T y$ : $X_a^T X_a = [[1, 1, 1], [1, 2, 3]] \cdot [[1, 1], [1, 2], [1, 3]] = [[3, 6], [6, 14]]$ and $X_a^T y = [[1, 1, 1], [1, 2, 3]] \cdot [2, 4, 6] = [12, 28]$
Next, we calculate the inverse of $X_a^T X_a$ : $(X_a^T X_a)^{-1} = [[3, 6], [6, 14]]^{-1} = \frac{1}{(3 \cdot 14 - 6 \cdot 6)} \cdot [[14, -6], [-6, 3]] = \frac{1}{6} \cdot [[14, -6], [-6, 3]] = [[\frac{7}{3}, -1], [-1, \frac{1}{2}]]$
Finally, we calculate the weight vector $w = (X_a^T X_a)^{-1} X_a^T y = [[\frac{7}{3}, -1], [-1, \frac{1}{2}]] \cdot [12, 28] = [0, 2]$

Editor

Python 3.13.1

0/0

Run code to see test results.

📘