PIXELBANKv8.2.1
Menu
Sign In

Feed-Forward Network

EasyTransformer Components

Implement a transformer feed-forward network (FFN).

The FFN applies two linear transformations with a ReLU activation in between: FFN(x)=ReLU(xW1+b1)W2+b2\text{FFN}(x) = \text{ReLU}(xW_1 + b_1)W_2 + b_2

For simplicity, use no biases (b1=0, b2=0). Use numpy seed 42 to generate W1 and W2.

Input:

  • Line 1: d d_ff (model dimension, FFN hidden dimension)
  • Line 2: space-separated floats (input vector of dimension d)

Output: Output vector of dimension d, rounded to 4 decimal places.

Generate weights: np.random.seed(42), W1 = np.random.randn(d, d_ff) * 0.1, W2 = np.random.randn(d_ff, d) * 0.1

Example:

Input:
2 4
1.0 -1.0
Output:
0.0107 -0.0510
Reasoning:
  • We generate the weights W1W_1 and W2W_2 using np.random.seed(42), with W1=np.random.randn(2,4)∗0.1W_1 = np.random.randn(2, 4) * 0.1 and W2=np.random.randn(4,2)∗0.1W_2 = np.random.randn(4, 2) * 0.1.
  • The input vector x=[1.0,−1.0]x = [1.0, -1.0] is then transformed by W1W_1 and passed through the ReLU activation function: ReLU(xW1)=ReLU([1.0,−1.0]⋅W1)ReLU(xW_1) = ReLU([1.0, -1.0] \cdot W_1).
  • The result is then transformed by W2W_2: ReLU(xW1)W2=ReLU([1.0,−1.0]⋅W1)⋅W2ReLU(xW_1)W_2 = ReLU([1.0, -1.0] \cdot W_1) \cdot W_2.
  • The final output is the result of this transformation, rounded to 4 decimal places: [0.0107,−0.0510][0.0107, -0.0510].

Constraints:

  • 1 <= d <= 10, 1 <= d_ff <= 20
  • Use np.random.seed(42) for weight initialization
  • Scale weights by 0.1
  • ReLU: max(0, x)
  • Round to 4 decimal places
Editor

Test Results

0/0
Run code to see test results.