PIXELBANKv8.2.1
Menu
Sign In

Multiple Linear Regression (Normal Equation)

MediumLinear Regression

Implement multiple linear regression using the normal equation.

Given a feature matrix XX (without bias column) and target vector yy, compute the weight vector ww that minimizes the mean squared error.

First, prepend a column of ones to XX to form the augmented matrix XaX_a, then solve: w=(XaTXa)−1XaTyw = (X_a^T X_a)^{-1} X_a^T y

Return the weight vector as a list, rounded to 4 decimal places. The first element is the bias/intercept.

You must implement matrix operations from scratch (transpose, multiply, invert).

Example:

Input:
X = [[1], [2], [3]]
y = [2, 4, 6]
Output:
[0.0, 2.0]
Reasoning:
  • First, we prepend a column of ones to XX to form the augmented matrix XaX_a: Xa=[[1,1],[1,2],[1,3]]X_a = [[1, 1], [1, 2], [1, 3]]
  • Then, we calculate XaTXaX_a^T X_a and XaTyX_a^T y: XaTXa=[[1,1,1],[1,2,3]]⋅[[1,1],[1,2],[1,3]]=[[3,6],[6,14]]X_a^T X_a = [[1, 1, 1], [1, 2, 3]] \cdot [[1, 1], [1, 2], [1, 3]] = [[3, 6], [6, 14]] and XaTy=[[1,1,1],[1,2,3]]⋅[2,4,6]=[12,28]X_a^T y = [[1, 1, 1], [1, 2, 3]] \cdot [2, 4, 6] = [12, 28]
  • Next, we calculate the inverse of XaTXaX_a^T X_a: (XaTXa)−1=[[3,6],[6,14]]−1=1(3⋅14−6⋅6)⋅[[14,−6],[−6,3]]=16⋅[[14,−6],[−6,3]]=[[73,−1],[−1,12]](X_a^T X_a)^{-1} = [[3, 6], [6, 14]]^{-1} = \frac{1}{(3 \cdot 14 - 6 \cdot 6)} \cdot [[14, -6], [-6, 3]] = \frac{1}{6} \cdot [[14, -6], [-6, 3]] = [[\frac{7}{3}, -1], [-1, \frac{1}{2}]]
  • Finally, we calculate the weight vector w=(XaTXa)−1XaTy=[[73,−1],[−1,12]]⋅[12,28]=[0,2]w = (X_a^T X_a)^{-1} X_a^T y = [[\frac{7}{3}, -1], [-1, \frac{1}{2}]] \cdot [12, 28] = [0, 2]

Constraints:

  • X is a 2D list (n samples x m features)
  • y is a 1D list of n targets
  • Return a list of (m+1) weights rounded to 4 decimal places
  • First weight is the intercept (bias term)
  • Implement matrix operations without numpy
Editor

Test Results

0/0
Run code to see test results.