📘

LoRA Forward Pass

EasyFine-Tuning Mechanics 📚 LLM 2: Training & Alignment

Implement the LoRA forward pass combining frozen weights with low-rank adaptation.

In LoRA, the output is: $y = xW + x\Delta W = xW + x(AB)$

where W is the frozen pretrained weight, and A, B are the LoRA matrices.

Input:

Line 1: d r (dimension, rank)
Line 2: space-separated floats (input vector x of dimension d)

Generate W, A, B with np.random.seed(42):

W = np.random.randn(d, d) * 0.1
A = np.random.randn(d, r) * 0.1
B = np.random.randn(r, d) * 0.01

Output: Output vector y = x @ W + x @ A @ B, rounded to 4 decimal places.

Example:

Input:

3 2
1.0 0.0 0.0

Output:

0.0495 -0.0132 0.0636

Reasoning:

We start by generating the frozen pretrained weight $W$ and LoRA matrices $A$ and $B$ using the given dimensions and np.random.seed(42): $W = np.random.randn(3, 3) * 0.1$ , $A = np.random.randn(3, 2) * 0.1$ , and $B = np.random.randn(2, 3) * 0.01$ .
Next, we compute the output vector $y$ by applying the LoRA forward pass formula: $y = xW + x\Delta W = xW + x(AB)$ , where $x = [1.0, 0.0, 0.0]$ .
We calculate $x @ W$ and $x @ A @ B$ separately, then add the results to obtain $y$ .
Finally, we round the elements of $y$ to 4 decimal places to obtain the output vector: $y = [0.0495, -0.0132, 0.0636]$ .

Constraints:

np.random.seed(42), then generate W, A, B in order
Output y = x @ W + x @ A @ B
Round to 4 decimal places

More from LLM 2: Training & Alignment

RLHF Reward ShapingHard Implicit Reward from DPOMedium PPO Clipped ObjectiveMedium DPO LossMedium KL Divergence PenaltyEasy

Background Knowledge

The problem is based on the concept of Low-Rank Adaptation (LoRA), a technique used in fine-tuning large pre-trained models. LoRA allows for efficient adaptation of a pre-trained model to a new task by adding low-rank matrices to the pre-trained weights. This approach reduces the number of parameters to be trained, making it more computationally efficient. The key idea is to represent the adaptation as a low-rank matrix, which can be factorized into two smaller matrices, A and B.

In the context of neural networks, LoRA is used to adapt the weights of a pre-trained model to a new task. The pre-trained weights are frozen, and the adaptation is represented as a low-rank matrix added to the pre-trained weights. This approach enables the model to learn task-specific features while leveraging the knowledge learned from the pre-training task. The forward pass is a crucial component of neural networks, where the input is propagated through the network to produce an output.

The mathematical formulation of LoRA is based on the idea of representing the adaptation as a low-rank matrix, which can be factorized into two smaller matrices, A and B. The output of the LoRA forward pass is given by the equation: $y = xW + x\Delta W = xW + x(AB)$ , where W is the frozen pre-trained weight, and A and B are the LoRA matrices. This equation represents the adaptation of the pre-trained weights to the new task.

Algorithm/Approach

The general approach to solving this problem involves implementing the LoRA forward pass using the given equation. The algorithm can be broken down into the following steps:

Generate the pre-trained weight W and the LoRA matrices A and B using the given formulas.
Compute the output y using the equation $y = xW + x\Delta W = xW + x(AB)$ .
Round the output y to 4 decimal places.

Step-by-Step Strategy

To implement the solution, follow these steps:

Read the input values d and r, which represent the dimension and rank of the matrices.
Generate the input vector x from the given space-separated floats.
Generate the pre-trained weight W and the LoRA matrices A and B using the given formulas and np.random.seed(42).
Compute the output y using the equation $y = xW + x\Delta W = xW + x(AB)$ .
Round the output y to 4 decimal places.

Common Pitfalls

When implementing the solution, watch out for the following:

Make sure to use the correct formulas to generate the pre-trained weight W and the LoRA matrices A and B.
Use the correct equation to compute the output y.
Round the output y to 4 decimal places as required.

Time & Space Complexity

The time complexity of the solution is O(d^2) due to the matrix multiplications, where d is the dimension of the matrices. The space complexity is also O(d^2) due to the storage of the matrices W, A, and B. Note that the rank r of the LoRA matrices A and B is typically much smaller than the dimension d, which reduces the computational cost of the matrix multiplications.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.