LoRA Adapter Parameter Count

easyPROEfficient Adaptation & Hallucination MetricsVision-Language ModelsVLM 1: Vision-Language Models

Problem Statement

LoRA freezes a weight matrix W (d_out x d_in) and learns a low-rank update B A with rank r. Count the trainable parameters the adapter adds.

Background

For a base matrix of shape (d_out, d_in), LoRA introduces A of shape (r, d_in) and B of shape (d_out, r), so the update W + BA adds

$r \cdot d_{in} + d_{out} \cdot r = r\,(d_{in} + d_{out})$

trainable parameters, versus d_out * d_in for full fine-tuning. That ratio is why a rank-8 adapter trains a fraction of a percent of the weights.

Your Task

Implement:

def lora_params(d_in, d_out, r):

Return the number of trainable LoRA parameters as an int.

Input Format

d_in, d_out, r (int).

Output Format

A single int.

Sample

print(lora_params(4096, 4096, 8))

Output:

Example

Input

print(lora_params(4096, 4096, 8))

Output

Reasoning

Identify the dimensions of the two low-rank matrices: matrix $A$ has shape $(r, d_{in})$ and matrix $B$ has shape $(d_{out}, r)$ , where the number of parameters in each is the product of its dimensions.
Calculate the parameter count for matrix $A$ using the input values $r=8$ and $d_{in}=4096$ : $8 \times 4096 = 32768$ .
Calculate the parameter count for matrix $B$ using the input values $d_{out}=4096$ and $r=8$ : $4096 \times 8 = 32768$ .
Sum the parameters from both matrices to get the total trainable parameters, which is equivalent to computing $r(d_{in} + d_{out})$ : $32768 + 32768 = 65536$ .
The final output is 65536

Constraints

1 <= d_in, d_out <= 100000, 1 <= r <= min(d_in, d_out).
Parameters are r*(d_in + d_out) (no biases).
Return an int.

More from VLM 1: Vision-Language Models

Visual Token Pruning by Attention Masshard CHAIR-i and CHAIR-s Hallucination Ratesmedium Merge a LoRA Update into the Base Weightmedium QLoRA Memory Footprint Estimatemedium POPE Precision, Recall, and F1medium

Concept primerPREVIEW

1. Background Knowledge

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique for large neural networks. Instead of updating every weight in a frozen matrix $W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ , LoRA learns a low-rank decomposition of the weight update $\Delta W$ . The update is factored into two smaller matrices: $A \in \mathbb{R}^{r \times d_{\text{in}}}$ and $B \in \mathbb{R}^{d_{\text{out}} \times r}$ , where $r \ll \min(d_{\text{in}}, d_{\text{out}})$ . The effective weight becomes $W + B A$ , and only $A$ and $B$ are trained.

The key insight is that a full-rank update requires $d_{\text{out}} \cdot d_{\text{in}}$ parameters, while the LoRA update requires only $r \cdot d_{\text{in}} + d_{\text{out}} \cdot r$ parameters. This is a massive reduction when $r$ is small (e.g., $r = 8$ ) relative to the matrix dimensions. For a $4096 \times 4096$ matrix, full fine-tuning needs $\approx 16.7$ million parameters, whereas a rank-8 LoRA adapter needs only $8 \times 4096 + 4096 \times 8 = 65{,}536$ parameters — roughly $0.4\%$ of the original.

This problem tests your understanding of how matrix dimensions translate into parameter counts. Each matrix contributes a number of trainable parameters equal to the product of its row and column dimensions. The total trainable parameter count is simply the sum of the parameter counts of all learned matrices.

2. Algorithm Approach

This is a direct computation problem. There is no search, iteration, or optimization involved. The approach is:

Identify the shapes of the two LoRA matrices: $A$ is $(r, d_{\text{in}})$ and $B$ is $(d_{\text{out}}, r)$ .
Compute the number of elements (parameters) in each matrix by multiplying its dimensions.
Sum the two counts to get the total trainable parameters.

The formula is:

\text{params} = r \cdot d_{\text{in}} + d_{\text{out}} \cdot r

This can also be factored as $r \cdot (d_{\text{in}} + d_{\text{out}})$ , which is algebraically equivalent and slightly more compact.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results