Paper breakdownContinual LearningFREE2026

Composing Continual Learning Mechanisms

Continual Learning Mechanisms Compose for Long-Horizon Memorization

Zheyuan Zhang, Alvin Zhang, Daniel Khashabi, Tianmin Shu

Read the Paper on arXiv

Concepts

Formulas

Figures

Demos

Paper Overview

Composing Continual Learning Mechanisms (Zhang, Zhang, Khashabi & Shu — Johns Hopkins University, 2026) asks a deceptively simple question: if a language model updates its own parameters to absorb new information, can it still remember what it learned a hundred updates ago? The authors call this long-horizon memorization, and they study it through continual supervised fine-tuning on a stream of 100 query-answer tasks — no stored examples from earlier tasks, no task identifier at inference. The model must learn each new task while holding onto every association it has already been taught.

The starting fact is bleak. After 100 tasks, naive sequential fine-tuning retains only 1.2% of what it learned. Catastrophic forgetting at this horizon is near-total, and — crucially — no single continual-learning mechanism the authors evaluate fixes it. Every standalone method (replay, distillation, EWC, LoRA tricks) is eliminated well before the 100-task mark.

The paper's thesis is that mechanisms addressing complementary sources of forgetting should work far better when composed than any of them alone. To make that testable, the authors organize the design space along two axes. Anchors answer what prior information each update should preserve: a data anchor (generative replay), a function anchor (self-distillation), and a weight anchor (importance-based regularization). Low-rank allocation rules answer where successive updates are retained: shared LoRA reuses one adapter, merged LoRA folds each update into the dense weights and reinitializes. Crossing these gives a large combinatorial space, which they navigate with task-level successive halving and then dissect with a 2⁴ factorial experiment.

The payoff is a clean composition result. The best method — all three anchors plus merged LoRA — is the only configuration that ranks among the top 3 on all three datasets, and it raises average final retention from 1.2% to 34.9%, a 28-fold improvement. The two biggest levers, the data anchor and merged LoRA, interact super-additively: together they buy far more than the sum of their parts.

Explainer videoThe whole paper, animated

Chapter Roadmap

Click any topic to jump in

Interactive demo

Long-Horizon Memorization

Learn 100 query-answer tasks by continual fine-tuning, no stored examples, no task IDs — and naive SFT forgets almost everything, retaining just 1.2%.

attacked by

Three Anchors

Data, function, and weight anchors each specify WHAT prior information an update must preserve — replay, self-distillation, importance-weighted regularization.

combined with

Low-Rank Allocation

Shared vs merged LoRA decides WHERE each update is retained — reuse one adapter, or fold it into the dense weights and start a fresh one every task.

searched via

Searching the Design Space

Task-level successive halving prunes 90 compositions down to 10 over longer horizons; a 2⁴ factorial then measures each mechanism's main and interaction effects.

measured by

Results

The best method combines all three anchors with merged LoRA — top 3 on every dataset, 34.9% average final retention, a 28-fold gain over naive fine-tuning.

interpreted as

What Composes, and What Doesn't

Replay and merged LoRA are the super-additive core; SI and SD help conditionally; and memory half-life is delayed, not defeated.

Concept by concept

6 sections, in reading order.

01 / 06Long-Horizon Memorization

Prompting and retrieval can hand a model new information at inference time, but that information lives outside the parameters and has to be supplied again on every call. This paper asks the opposite question: can repeated parameter updates build and preserve memory inside the model itself? For parameters to serve as memory, the model must still recall an association after many more updates have landed on top of it.

The setting is continual supervised fine-tuning in the domain-incremental regime of Van de Ven and Tolias. A stream of $T = 100$ tasks arrives one at a time. Task $t$ contains query-answer pairs $D_t$ ; when it arrives the model inherits the previous state $\Theta_{t-1}$ but may not revisit raw examples from earlier tasks, and at inference it is never told which task a query belongs to. Because the study is about memorization rather than generalization, each task is evaluated on exactly the examples it was trained on — the question is pure recall, not transfer.

The obstacle is catastrophic forgetting: an update that teaches task $t$ overwrites parameters that encoded earlier associations. The tension the whole paper lives inside is the stability-plasticity trade-off — stay plastic enough to learn the new task, stay stable enough not to erase the old ones. At a 100-task horizon this is brutal. Naive fine-tuning has a memory half-life of a single task on two of the three datasets: an association is more likely than not to be gone by the very next update. Final retention after all 100 tasks sits at 1.2%.

What makes long-horizon memorization a distinct continual-learning problem — and not just a rerun of image-classification benchmarks — is the combination of a long horizon, no task identity, no rehearsal buffer of real data, and a recall (rather than transfer) objective. Existing language benchmarks mostly measure transfer across heterogeneous downstream tasks or changing corpora; sequential model-editing studies targeted edits. None isolates this question, so the authors build the evaluation from scratch.

Key Points

1
Long-horizon memorization: learn 100 query-answer tasks by continual SFT, retaining associations across all of them — memory inside the parameters, not supplied at inference like prompting or retrieval
2
Domain-incremental setting: no task identifier is given at inference, and raw examples from earlier tasks may not be revisited
3
It is memorization, not generalization — each task is evaluated on the same examples it was trained on, so the metric is pure recall
4
Catastrophic forgetting is the core obstacle: each new update overwrites parameters that stored earlier associations (the stability-plasticity trade-off)
5
Naive sequential fine-tuning retains only 1.2% after 100 tasks, with a memory half-life of ~1 task on Symbol-QA and LLM-QA
6
No single mechanism the authors evaluate maintains strong retention at this horizon — which motivates composing complementary mechanisms

Mathematical Formulation

Current-Task SFT Loss

$\mathcal{L}^{t}_{\text{SFT}}(\Theta) \;=\; -\,\mathbb{E}_{(x,y)\sim D_t}\big[\log p_\Theta(x, y)\big]$

The standard likelihood objective for task $t$ . Query tokens are not masked — the authors argue it is hard to separate query from answer in applications like test-time training, so the whole sequence $(x, y)$ is scored. This term drives plasticity; the anchors that follow supply stability.

Key Points

1
An anchor specifies what prior information each update must preserve; the three anchors add three retention terms to the SFT loss
2
Data anchor — generative replay: a frozen previous model generates 300 pseudo-sequences per task from one replay token; each current minibatch is paired with a replay minibatch, with soft next-token targets (heritage: Deep Generative Replay, LAMOL)
3
Function anchor — self-distillation: the previous model is a teacher; penalize divergence on current-task inputs (Learning without Forgetting). Unlike the data anchor, it distills on real current data, not replayed sequences
4
Weight anchor — importance regularization: a quadratic penalty that discourages changing parameters important to earlier tasks (EWC, Online EWC, SI)
5
EWC/Online EWC use diagonal Fisher information; SI estimates importance from contributions accumulated along the optimization path
6
Three anchors constrain the update along orthogonal axes — inputs, functions, and parameters — so they address complementary sources of forgetting

Mathematical Formulation

Composite Continual Objective

$\Theta_t \;=\; \arg\min_{\Theta}\; \mathcal{L}^{t}_{\text{SFT}}(\Theta) \;+\; R^{D}_{t}(\Theta) \;+\; R^{F}_{t}(\Theta) \;+\; R^{W}_{t}(\Theta)$

The SFT loss plus the three anchor penalties: $R^{D}$ (data), $R^{F}$ (function), $R^{W}$ (weight). Composing mechanisms means turning individual terms on together — the factorial experiment later toggles exactly these.

Data Anchor (Generative Replay)

$R^{D}_{t}(\Theta) \;=\; \mathbb{E}_{z\sim Q_{t-1}}\big[\ell_D(\Theta, z)\big]$

$Q_{t-1}$ is the distribution of replay sequences generated by the frozen previous model. Vanilla and generative replay differ in how $Q_{t-1}$ is built; hard vs soft replay differ in the loss $\ell_D$ (this paper uses soft next-token targets).

Function Anchor (Self-Distillation)

$R^{F}_{t}(\Theta) \;=\; \mathbb{E}_{x\sim\mu_t}\big[\,d\big(q_{t-1}(\cdot \mid x),\; p_\Theta(\cdot \mid x)\big)\big]$

On current-task inputs $x\sim\mu_t$ , the divergence $d$ between the previous model's reference distribution $q_{t-1}$ and the current model's output $p_\Theta$ is penalized — the previous model teaches the new one to keep its old input-output mapping.

Weight Anchor (Importance Regularization)

$R^{W}_{t}(\Theta) \;=\; \tfrac{1}{2}\,\big(\vartheta - \vartheta^{\star}_{t-1}\big)^{\top} H_{t-1}\,\big(\vartheta - \vartheta^{\star}_{t-1}\big),\qquad H_{t-1}\succeq 0$

A quadratic that pulls tracked parameters $\vartheta$ back toward their pre-task values $\vartheta^{\star}_{t-1}$ , weighted by accumulated importance $H_{t-1}$ . EWC uses diagonal Fisher information; SI accumulates importance along the training path.

Key Points

1
Every task is adapted with LoRA: $W_0 + \rho BA$ with rank- $r$ matrices $A, B$ and scale $\rho = \alpha_{\text{LoRA}}/r$
2
Shared LoRA: keep optimizing the same $A, B$ across all tasks — one adapter accumulates every update, so tasks share (and fight over) one low-rank workspace
3
Merged LoRA: after each task, fold the adapter into the dense weights and reinitialize a fresh $A, B$ (and optimizer) — adapts ReLoRA's merge-and-reinit to continual learning
4
Both rules keep constant retained state (one dense model + one LoRA pair per matrix), independent of task count — a deliberate scope choice
5
State-growing alternatives — O-LoRA (orthogonal per-task subspaces) and sequential OSRM — are tested separately and do not consistently improve over merged LoRA
6
Merged LoRA gives each task a clean workspace and consolidates the past into dense weights the current gradients no longer directly perturb

Mathematical Formulation

Shared vs Merged LoRA

$W_t \;=\; \begin{cases} W_0 + \rho\,B_t A_t, & \text{shared LoRA} \\[4pt] W_{t-1} + \rho\,B_t A_t, & \text{merged LoRA} \end{cases}$

Shared LoRA always adapts the original $W_0$ with the same accumulating adapter. Merged LoRA adapts $W_{t-1}$ — the dense weight that already absorbed every previous task — with a freshly initialized adapter, so each task begins from a clean low-rank workspace.

Key Points

1
The initial search space is 90 configurations ( $3 \times 3 \times 5 \times 2$ ) per dataset per seed — too many to train fully to 100 tasks
2
Task-level successive halving (TSH) allocates more sequential tasks (not more iterations) to promising configs: 90 → 45 (after 10 tasks) → 23 (20) → 10 (50) → 100 tasks
3
10-task rankings agree strongly with the final 100-task rankings even under a different task order, so early retention is a reliable pruning signal
4
SD₁/SD₂ are self-distillation weights 1 and 3; Replay₁–₄ cross replay weights {0.5, 0.75} with generation temperatures {1.0, 1.5}
5
A 2⁴ factorial over {SI, SD, Replay, Merge} then measures each mechanism's main effect and all interaction effects cleanly
6
Two immediate findings: no standalone mechanism reaches 50 tasks, and every method reaching 100 tasks combines a data anchor with merged LoRA

Mathematical Formulation

Horizon-r Retention Score

$F_r(a) \;=\; \frac{1}{|S|}\sum_{s\in S}\; \frac{1}{r}\sum_{j=1}^{r} M^{a,s}_{r,j}$

For configuration $a$ and seed set $S$ , $F_r(a)$ is mean final retention after $r$ tasks, averaged over seeds. $M^{a,s}_{r,j}$ is the accuracy on task $j$ after learning tasks $1$ through $r$ . TSH keeps the highest-scoring configs at each horizon.

Initial Search Space

$n_1 = |A_1| = 3 \times 3 \times 5 \times 2 = 90$

Three weight-anchor options ( $\varnothing$ , online EWC, SI), three function options ( $\varnothing$ , SD₁, SD₂), five data options ( $\varnothing$ plus four replay settings), and two allocation rules — ninety compositions before any pruning.

Key Points

1
Three datasets, increasing realism: Symbol-QA (10k random key-value pairs), LLM-QA (10k fictional facts), Real-QA (5k filtered natural QA pairs) — each 100 tasks
2
Best standalone: only 4.2% / 7.5% / 12.5% retention on Symbol-QA / LLM-QA / Real-QA after 100 tasks
3
Best compositions: 23.2% / 41.8% / 54.8% on the same datasets — dramatically higher
4
All three anchors + merged LoRA is the only composition in the top 3 on every dataset; averaged, it hits 34.9% vs 1.2% for naive fine-tuning — a 28-fold gain
5
Immediate acquisition is near-perfect for these methods, so the gap is forgetting, not learning difficulty
6
Memory half-life: naive 1/1/2 tasks → best standalone 4/6/11 → strongest composition 19/32/44 tasks
7
Caveat: better memorization does not preserve general capability — all methods degrade on GSM8K/MATH/MGSM/MMLU-Redux; O-LoRA preserves more general ability than merged LoRA

Mathematical Formulation

Final Retention, Immediate Acquisition, Forgetting

$\text{Final} = \frac{1}{T}\sum_{j=1}^{T} M_{T,j}, \quad \text{Diag} = \frac{1}{T}\sum_{j=1}^{T} M_{j,j}, \quad \text{Forget} = \frac{1}{T-1}\sum_{j=1}^{T-1}\Big(\max_{j\le i\le T} M_{i,j} - M_{T,j}\Big)$

From the temporal accuracy matrix $M_{i,j}$ (accuracy on task $j$ after learning through task $i$ ): Final is mean accuracy after the last task, Diag is accuracy right after each task is learned, and Forget is the average drop from each task's peak to its final accuracy (excluding the last task, which nothing later can erase).

Memory Half-Life

$R(a) = \text{mean}\{\,M_{i,j} : i - j = a\,\}, \qquad t_{1/2} = \min\{\,a : R(a) < \tfrac{1}{2} R(0)\,\}$

$R(a)$ averages accuracy over all task pairs a memory age $a$ apart. The half-life $t_{1/2}$ is the first age at which retention drops below half its fresh value — 1–2 tasks for naive fine-tuning, up to 44 for the strongest composition.

Key Points

1
Replay and merged LoRA have the two largest main effects on every dataset (replay +9.5 to +19.3 pts; merged LoRA +5.9 to +20.5 pts)
2
Their interaction is positive and significant on all three datasets — the headline super-additive effect
3
Super-additivity: standalone gains sum to 3.9 / 7.7 / 13.9 pts, but combined they lift retention by 15.6 / 31.0 / 46.9 pts — 3–4× more than the sum of parts
4
SD is conditional: positive main effect everywhere, but a negative interaction with replay on the natural-language datasets (they overlap)
5
SI is conditional: helps on natural-language data, no effect on Symbol-QA, and interacts negatively with merged LoRA — because merged LoRA reinitializes the factors SI's importance values were tied to
6
Composition delays, it does not defeat: every retention curve still declines with memory age, and general capability still degrades — an order of magnitude more time, not permanence

Mathematical Formulation

Super-Additive Interaction

$\Delta_{\text{combined}} \;\gg\; \Delta_{\text{Replay}} + \Delta_{\text{Merge}}, \qquad \text{e.g. } 46.9 \;\gg\; 13.9 \text{ (Real-QA)}$

The gain from combining replay and merged LoRA far exceeds the sum of their standalone gains — the definition of a super-additive (positive) interaction. On Real-QA the pair delivers 46.9 points where the parts predict only 13.9.

Go deeper

Study Plans

LLM Study Plan ML Study Plan All Papers Practice Problems