MLP Forward Pass
Implement the forward pass of a two-layer Multi-Layer Perceptron (MLP).
Given input x, two weight matrices W1, W2 and two bias vectors b1, b2:
- Hidden layer: h=ReLU(W1⋅x+b1)
- Output layer: o=W2⋅h+b2
Where ReLU(z)=max(0,z) applied element-wise.
Return the output vector, rounded to 4 decimal places.
Example:
x = [1, 2] W1 = [[0.5, 0.3], [-0.2, 0.8]] b1 = [0.1, -0.1] W2 = [[0.4, -0.5]] b2 = [0.2]
[0.03]
- First, we calculate the hidden layer h using the given input x, weight matrix W1, and bias vector b1: h=ReLU(W1⋅x+b1)=ReLU([0.5−0.20.30.8]⋅[12]+[0.1−0.1])
- We perform the matrix multiplication and addition: h=ReLU([0.5∗1+0.3∗2−0.2∗1+0.8∗2]+[0.1−0.1])=ReLU([0.5+0.6+0.1−0.2+1.6−0.1])=ReLU([1.21.3])
- Applying the ReLU function: h=[max(0,1.2)max(0,1.3)]=[1.21.3]
- Then, we calculate the output o using the hidden layer h, weight matrix W2, and bias vector b2: $o = W_2 \cdot h + b_2 = \begin{bmatrix} 0.4 & -0.5 \end{bmatrix} \cdot \begin{bmatrix} 1.2 \ 1.3 \end{bmatrix} + \begin{bmatrix} 0.2 \end{bmatrix} = \begin{bmatrix} 0.41.2 - 0.51.3 + 0.2 \end{bmatrix} = \begin{bmatrix}
Constraints:
- x: 1D list (input vector, d_in features)
- W1: 2D list (d_hidden x d_in), b1: 1D list (d_hidden)
- W2: 2D list (d_out x d_hidden), b2: 1D list (d_out)
- Return 1D list of output values rounded to 4 decimal places
Background Knowledge
The Multi-Layer Perceptron (MLP) is a type of neural network that consists of multiple layers of interconnected nodes or "neurons." Each layer receives input from the previous layer, performs a computation on that input, and then sends the output to the next layer. The forward pass refers to the process of passing input through the network, layer by layer, to produce an output. In this problem, we're dealing with a two-layer MLP, which means we have an input layer, a hidden layer, and an output layer.
The hidden layer uses the ReLU (Rectified Linear Unit) activation function, which applies the max(0,z) function element-wise to the input. This means that any negative values in the input will be set to 0, while positive values will remain unchanged. The ReLU function is commonly used in neural networks because it's simple to compute and helps to introduce non-linearity into the model. The output layer uses a linear activation function, which means that the output is simply a weighted sum of the inputs, plus a bias term.
In order to implement the forward pass of an MLP, we need to have a good understanding of linear algebra and matrix operations. Specifically, we need to know how to perform matrix multiplication and addition, as well as how to apply element-wise functions like ReLU. We also need to understand how to represent the weights and biases of the network as matrices and vectors, and how to use these representations to compute the output of each layer.
Algorithm/Approach
The general approach to solving this problem involves the following steps:
- Compute the output of the hidden layer by multiplying the input by the weight matrix, adding the bias vector, and applying the ReLU activation function.
- Compute the output of the output layer by multiplying the hidden layer output by the weight matrix, adding the bias vector, and applying the linear activation function.
- Return the output of the output layer, rounded to 4 decimal places.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.