PIXELBANKv8.2.1
Menu
Sign In

Rotary Position Embedding

HardEmbeddings & Positional Encoding

Implement Rotary Position Embedding (RoPE).

RoPE rotates pairs of embedding dimensions by position-dependent angles. For a vector x of dimension D at position pos:

  • Split x into pairs: (x[0], x[1]), (x[2], x[3]), ...
  • For pair i, compute angle θ_i = pos / 10000^(2i/D)
  • Apply rotation: x'[2i] = x[2i] * cos(θ_i) - x[2i+1] * sin(θ_i) x'[2i+1] = x[2i] * sin(θ_i) + x[2i+1] * cos(θ_i)

Input:

  • Line 1: pos D (position, dimension)
  • Line 2: space-separated floats (the vector)

Output: Rotated vector, values rounded to 4 decimal places.

Example:

Input:
1 4
1.0 0.0 1.0 0.0
Output:
0.5403 0.8415 0.9999 0.0100
Reasoning:
  • The input vector is split into pairs: (1.0,0.0)(1.0, 0.0) and (1.0,0.0)(1.0, 0.0).
  • For each pair ii, the angle θi\theta_i is computed: θ0=1100002/4=1100\theta_0 = \frac{1}{10000^{2/4}} = \frac{1}{100} and θ1=1100004/4=110000\theta_1 = \frac{1}{10000^{4/4}} = \frac{1}{10000}.
  • The rotation is applied to each pair:
    • For the first pair: x′[0]=1.0⋅cos⁡(1100)−0.0⋅sin⁡(1100)x'[0] = 1.0 \cdot \cos(\frac{1}{100}) - 0.0 \cdot \sin(\frac{1}{100}) and x′[1]=1.0⋅sin⁡(1100)+0.0⋅cos⁡(1100)x'[1] = 1.0 \cdot \sin(\frac{1}{100}) + 0.0 \cdot \cos(\frac{1}{100}), resulting in approximately 0.54030.5403 and 0.84150.8415 respectively,
    • For the second pair: x′[2]=1.0⋅cos⁡(110000)−0.0⋅sin⁡(110000)x'[2] = 1.0 \cdot \cos(\frac{1}{10000}) - 0.0 \cdot \sin(\frac{1}{10000}) and x′[3]=1.0⋅sin⁡(110000)+0.0⋅cos⁡(110000)x'[3] = 1.0 \cdot \sin(\frac{1}{10000}) + 0.0 \cdot \cos(\frac{1}{10000}), resulting in approximately 0.99990.9999 and 0.01000.0100 respectively.
  • The final output is the rotated vector: 0.5403,0.8415,0.9999,0.01000.5403, 0.8415, 0.9999, 0.0100.

Constraints:

  • 0 <= pos <= 100, 2 <= D <= 16 (D is even)
  • Use base 10000 for frequency
  • Round to 4 decimal places
Editor

Test Results

0/0
Run code to see test results.