PIXELBANKv8.2.1
Menu
Sign In

Layer Normalization

EasyTransformer Components

Implement layer normalization for a single vector.

Layer normalization normalizes across the feature dimension: LayerNorm(x)=x−μσ2+ϵ⋅γ+β\text{LayerNorm}(x) = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \cdot \gamma + \beta

where μ and σ² are the mean and variance of x, γ (gain) and β (bias) are learnable parameters.

For this problem, use γ = 1 and β = 0 (no affine transform), and ε = 1e-5.

Input: Space-separated floats (the vector x) Output: Normalized vector, values rounded to 4 decimal places.

Example:

Input:
1.0 2.0 3.0
Output:
-1.2247 0.0000 1.2247
Reasoning:
  • The input vector xx is [1.0,2.0,3.0][1.0, 2.0, 3.0].
  • We calculate the mean μ\mu and variance σ2\sigma^2 of xx: μ=1.0+2.0+3.03=2.0\mu = \frac{1.0 + 2.0 + 3.0}{3} = 2.0 and σ2=(1.0−2.0)2+(2.0−2.0)2+(3.0−2.0)23=23\sigma^2 = \frac{(1.0-2.0)^2 + (2.0-2.0)^2 + (3.0-2.0)^2}{3} = \frac{2}{3}.
  • Then, we apply the layer normalization formula: LayerNorm(x)=x−μσ2+ϵ⋅γ+β=x−2.023+1e−5⋅1+0\text{LayerNorm}(x) = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \cdot \gamma + \beta = \frac{x - 2.0}{\sqrt{\frac{2}{3} + 1e-5}} \cdot 1 + 0.
  • The final output is calculated by applying the formula to each element of xx: [1.0−2.023+1e−5,2.0−2.023+1e−5,3.0−2.023+1e−5]\left[\frac{1.0-2.0}{\sqrt{\frac{2}{3} + 1e-5}}, \frac{2.0-2.0}{\sqrt{\frac{2}{3} + 1e-5}}, \frac{3.0-2.0}{\sqrt{\frac{2}{3} + 1e-5}}\right] = [−1.2247,0.0000,1.2247][-1.2247, 0.0000, 1.2247].

Constraints:

  • 1 <= dimension <= 100
  • Use ε = 1e-5 for numerical stability
  • γ = 1, β = 0
  • Round to 4 decimal places
Editor

Test Results

0/0
Run code to see test results.