TinyLoRA

In standard LoRA, each module (Q, K, V, O attention projections plus MLP up/down/gate matrices) gets its own independent adapter with rank $r \geq 1$. For a model with $n_m$ modules, the minimum parameter count is:

$\text{Min params (LoRA)} = n_m \cdot (d + k) \cdot r \geq n_m \cdot (d + k)$

For Qwen2.5-7B with $d = k = 4096$, 80 transformer layers, and 7 modules per layer, $n_m = 560$. At rank $r = 1$, the minimum is $560 \times 8{,}192 = 4.59M$ parameters. LoRA-XS reduces this to $r^2$ per module via SVD, but at $r = 2$ that's still $4 \times 560 = 2{,}240$ parameters.

The constraint is architectural: each module needs its own independent adapter matrix, and the minimum size of that matrix is $r \times r = 4$ (at $r = 2$). Even if each adapter is tiny, the total scales linearly with $n_m$. With 560 modules, the parameter floor is in the thousands.

The concept of "rank" itself imposes this limit. A rank-$r$ update modifies the weight matrix along $r$ independent directions. Rank 1 means one direction per module — 560 directions total. But what if the task requires far fewer independent directions of modification across the entire model?

TinyLoRA introduces weight tying across modules. Instead of separate adapters, all $n_m = 560$ module positions share the same trainable vector $\mathbf{v} \in \mathbb{R}^u$:

• Full weight tying ($n_{\text{tie}} = n_m = 560$): All 560 positions use the identical vector $\rightarrow$ total parameters $= u$
• With $u = 13$: The entire model adaptation is controlled by 13 numbers
• The effective per-module rank is $u / n_m = 13 / 560 \approx 0.023$ — far below the rank-1 floor of any prior method

Each position still produces a different weight update $\Delta W_m$ because:

1. The fixed random matrices $P_i^{(m)}$ are drawn independently for each module $m$, so $R_m = \sum v_i P_i^{(m)}$ differs at every position
2. The SVD factors $U_m, \Sigma_m, V_m$ are computed from each module's unique frozen weight matrix $W_m$
3. The final update $\Delta W_m = U_m \Sigma_m R_m V_m^\top$ is therefore unique even though $\mathbf{v}$ is shared

The shared vector $\mathbf{v}$ acts as a global "style knob" — 13 continuous dials that are interpreted differently by each module's fixed projections. Turning dial $v_3$ might increase attention to certain token patterns in the Q-projection of layer 12 while simultaneously adjusting the MLP gating behavior in layer 45, because those two modules have completely different random bases $P_3^{(Q_{12})}$ and $P_3^{(\text{MLP}_{45})}$.

Tying granularity ablation reveals a smooth tradeoff:

Tying level	Total params	GSM8K (RL)
No tying ($n_{\text{tie}} = 1$)	$u \times 560$	Higher (more capacity)
Per-layer ($n_{\text{tie}} = 7$)	$u \times 80$	Slightly higher
Full tying ($n_{\text{tie}} = 560$)	$u = 13$	91% (sufficient for GSM8K)

The result challenges the assumption that model adaptation requires proportional capacity: 13 well-placed parameters suffice because the pretrained model already "knows" how to reason — it just needs a minimal directional nudge.

A natural question arises: can any training method achieve strong performance with 13 parameters, or does the training paradigm matter?

The base Qwen2.5-7B model scores 78% on GSM8K without any fine-tuning (using chain-of-thought prompting). Testing with supervised fine-tuning (SFT) on the same task reveals a stark gap:

Parameters	SFT Accuracy	Gain over base
13	83%	+5%
120	84%	+6%
1,000	~85%	+7%

SFT saturates quickly — adding 10x more parameters (13 → 120) yields only 1% improvement. The problem is fundamental to how SFT's loss function interacts with tiny capacity:

SFT computes cross-entropy loss on every token in the chain-of-thought solution. A typical GSM8K solution is 150-300 tokens containing: (1) problem restatement, (2) step-by-step reasoning with arithmetic, (3) intermediate variable assignments, (4) formatting tokens (newlines, "Step 1:", etc.), and (5) the final numerical answer. The 13-parameter adapter must simultaneously adjust its values to improve predictions across all these token positions. Since formatting tokens outnumber reasoning-critical tokens by roughly 3:1, the gradient signal is dominated by superficial pattern matching rather than mathematical correctness.

Furthermore, SFT requires the model to produce one specific chain-of-thought — but there are many valid reasoning paths to the correct answer. The 13 parameters cannot encode the specific surface form of any single path.

RL (GRPO) at 13 parameters achieves 91% — a 13-point gain over the 78% baseline, compared to SFT's 5-point gain. The performance gap is even starker at higher parameter counts:

Parameters	SFT	RL (GRPO)	RL advantage
13	83%	91%	+8%
120	84%	95%	+11%
1,000	~85%	~96%	+11%

The key insight is about signal density and dimensionality:

SFT provides a high-dimensional signal: each of the ~200 tokens in a solution contributes a gradient direction. With 13 trainable parameters, these ~200 gradient directions must be compressed into 13 dimensions — the resulting update is an average that washes out task-relevant signal. The adapter learns to make tiny improvements to the model's next-token prediction averaged over formatting, reasoning, and answer tokens — but cannot specialize.

RL provides a low-dimensional signal: only the final answer's correctness matters, producing a single scalar reward $r \in \{0, 1\}$. This binary signal has a clear interpretation: "this direction in the 13-dimensional parameter space produces more correct answers." The gradient from GRPO's advantage-weighted policy update points directly along the useful direction without dilution from irrelevant token positions.

The resampling mechanism amplifies this further: GRPO samples $K = 4$ solutions per problem. Across these samples, the correct reasoning patterns (appropriate arithmetic operations, correct variable tracking) remain consistent in solutions that earn reward, while irrelevant variation (exact wording, step ordering, formatting choices) differs randomly between samples. The group-relative advantage computation $A_i = (r_i - \bar{r}) / \sigma_r$ effectively performs a noise-cancellation operation — averaging out solution-specific artifacts and isolating the reward-correlated signal.

The authors frame this through an information-theoretic lens: RL's reward signal has low intrinsic dimensionality (essentially 1D: correct vs incorrect), which can be faithfully captured by a 13-dimensional parameter space. SFT's token-level signal has high intrinsic dimensionality (~200D), which cannot be compressed into 13 dimensions without catastrophic information loss.

Standard policy gradient methods like PPO require a separate value (critic) network to estimate the baseline for advantage computation. This value network typically has millions of parameters of its own — an absurd overhead when the policy adapter has only 13. Moreover, the value network must be trained concurrently, introducing additional optimization instability.

When the entire trainable adapter is a 13-dimensional vector $\mathbf{v} \in \mathbb{R}^{13}$, the optimization landscape is a 13-dimensional surface. This is simultaneously a blessing (low-dimensional = few local minima, smooth gradients) and a curse (extremely limited expressiveness, so the gradient must point in exactly the right direction).

The credit assignment problem is severe. Math reasoning requires generating long chains of thought — a typical GSM8K solution spans 150-300 tokens of arithmetic steps — where only the final numerical answer determines correctness. The reward is delayed by hundreds of autoregressive steps: which of the 13 parameters should be adjusted, and by how much, based on whether a 200-token reasoning chain produced the right number? Standard REINFORCE would produce extremely high-variance gradients in this setting.

GRPO (Group Relative Policy Optimization), introduced by DeepSeek, addresses all these challenges with an elegant design:

Algorithm:

1. Sample $K = 4$ solutions per problem from the current policy $\pi_{\theta}$
2. Score each with exact-match reward on the final numerical answer: $r_i \in \{0, 1\}$
3. Compute group advantage: $A_i = \frac{r_i - \bar{r}}{\sigma_r + \epsilon}$ where $\bar{r}$ and $\sigma_r$ are the mean and standard deviation within the group of $K$ solutions for that specific problem
4. Update policy using the clipped advantage-weighted log probability gradient (similar to PPO's clipped objective, but without a value network)

Why GRPO works for TinyLoRA:

The group-relative advantage is the key innovation. By comparing solutions to the same problem, GRPO eliminates problem difficulty as a confound: a correct solution to an easy problem (where 3 out of 4 samples are correct) receives a smaller advantage than a correct solution to a hard problem (where only 1 out of 4 is correct). This normalizes the reward signal across problems of different difficulty.

For TinyLoRA's 13-dimensional parameter space, this produces an exceptionally clean gradient. Consider what happens within one group of $K = 4$ solutions to the same problem:

• Solutions that get the right answer share certain characteristics (correct arithmetic operations, proper variable tracking)
• Solutions that fail differ in specific ways (wrong multiplication, lost intermediate values)
• The advantage weighting amplifies the parameter directions that correlate with correctness
• Across many such groups, the noise from problem-specific variation averages out

No value network needed: GRPO uses the group mean $\bar{r}$ as the baseline instead of a learned value function. This eliminates millions of extra parameters and the instability of concurrent critic training.

Training configuration:

• Dataset: GSM8K training set (7,473 problems)
• Epochs: 3 complete passes over the dataset
• Batch size: 64 problems per batch ($\times K = 4$ samples each = 256 generations per batch)
• Max sequence length: 4,096 tokens per generation
• Learning rate: swept from $10^{-7}$ to $2 \times 10^{-4}$ (optimal varies by parameter count)
• KL penalty: standard coefficient to prevent the adapter from pushing the policy too far from the base model
• Optimizer: AdamW applied to the 13-dimensional vector $\mathbf{v}$
• Total training cost: approximately 2-3 GPU-hours on a single A100 (the adapter is so small that the dominant cost is generation, not gradient computation)

A critical question for practical deployment: does TinyLoRA's extreme efficiency scale to different model sizes? If it only works for Qwen2.5-7B, the finding is interesting but limited in scope.

Several concerns arise:

• Smaller models might lack the latent reasoning capability that TinyLoRA activates — if mathematical reasoning isn't encoded in the pretrained weights, no 13-parameter adapter can create it from nothing
• Larger models have more modules to adapt ($n_m$ grows with depth and width), so they might need proportionally more parameters to modify all relevant modules
• The base model accuracy varies significantly: Qwen2.5-3B scores ~65% on GSM8K, 7B scores ~78%, and 14B scores ~84% without any fine-tuning. If TinyLoRA just provides a small multiplicative boost, the gains would be bounded by the base model's starting point

The scaling behavior of parameter-efficient methods has broad implications: if the minimum parameter count for effective adaptation decreases with model scale, this would fundamentally change how we think about customizing and deploying foundation models.

The paper reveals a clear inverse scaling law: larger models need fewer TinyLoRA parameters to reach the same fraction of their peak performance.

Parameters needed to reach 95% of peak GSM8K accuracy (with GRPO):

Model	Base accuracy	Peak (full RL)	Params for 95% of peak	Effective rank
Qwen2.5-3B	~65%	~88%	~350	~0.63
Qwen2.5-7B	~78%	~96%	~120	~0.21
Qwen2.5-14B	~84%	~97%	<80	<0.08

The trend is clear: the minimum parameter count decreases as model size increases, even though larger models have more modules. This strongly suggests that larger models have more latent reasoning capability already encoded in their pretrained weights. TinyLoRA's role is not to teach reasoning from scratch but to activate existing capabilities with a minimal directional nudge — and the larger the model, the smaller the nudge needed.

Interpretation — the "activation" hypothesis: The pretrained model already contains the circuits for mathematical reasoning, developed during pretraining on vast corpora. These circuits are not perfectly aligned for GSM8K-style problems — they need a small adjustment to their activation patterns. The 13 TinyLoRA parameters shift the model's behavior in the relevant direction, and this shift is more effective in larger models because their pretrained circuits are more complete and robust.

Bit-constrained regime: An additional finding reveals that when comparing formats at constant byte budget, fp32 outperforms bf16 byte-for-byte:

• 26 bytes of bf16 = 13 parameters → 91% accuracy
• 26 bytes of fp32 = ~6.5 parameters → slightly better accuracy than 13 bf16 parameters
• 52 bytes of fp32 = 13 parameters → 93% accuracy (higher than 13 bf16)

This is because each fp32 value has 23 mantissa bits vs bf16's 7, providing 3.3x more precision per parameter. In the ultra-low-dimensional regime, the optimizer needs to find a precise direction in parameter space — having fewer parameters with higher precision outperforms having more parameters with lower precision. The optimal direction matters more than the number of dimensions.

Extrapolation: If the scaling trend continues, models in the hundreds-of-billions parameter range might need only single-digit parameters for task adaptation — perhaps even a single fp64 parameter (8 bytes) encoding a direction in a 1-dimensional subspace projected through random matrices across thousands of modules. The adapter would literally be a single number.