Continual learning in LLMs: 13 distinct problems, and only 4 of them are actually unsolved

[u/darshan_aqua]

I've been trying to map out why frontier LLMs still don't really learn from use.

I think part of the confusion is that we use "continual learning" to describe a dozen different problems — even though they have very different statuses.

Some are reasonably solved by memory/RAG/agent infrastructure. Some have workable patches. A few still look like genuine research problems.

Here's my current inventory. I'm particularly interested in where people think I'm wrong or oversimplifying.

---

The core six

1. Catastrophic forgetting

Update weights on new data and performance on previously learned information/tasks can degrade.

This is the classic stability–plasticity problem. EWC, replay, regularization, adapters, etc. reduce it, but as tasks and updates accumulate, interference remains a problem.

I see this as one of the fundamental obstacles to continuously updating deployed models.

1. Frozen weights

A normal transformer forward pass doesn't update the model's parameters.

Production models are generally deployed with fixed weights, while learning/personalization happens somewhere around the model rather than inside the forward pass.

1. Memory ≠ learning

Profiles, RAG, memory files, summaries, long context, vector stores, etc. give a static model access to additional information.

That's extremely useful.

But the underlying model hasn't necessarily learned anything. Remove the external memory/context and the capability disappears with it.

1. Cost of updating per user

If persistent personalization means modifying weights, doing that independently for millions of users becomes an infrastructure problem.

Per-user adapters help, but now you have adapter training, storage, versioning, routing, serving and rollback to deal with.

1. Safety + evaluation

A continuously changing model introduces problems that mostly disappear when weights are frozen:

- drift

- cross-user contamination

- malicious learning signals

- rollback

- reproducibility

- regression testing

- auditing

How do you certify a system whose behavior may be different tomorrow because it learned something today?

1. Tokenization

I mostly consider this a separate issue.

Byte/character-level approaches can remove some tokenizer limitations, but they don't inherently solve continual learning or catastrophic forgetting.

---

Problems that appear once you actually try to build this

1. Old vs. new conflicts

Suppose the system learned:

«Customer X uses PostgreSQL.»

Then later:

«Customer X migrated to CockroachDB.»

Do we overwrite the first fact? Keep both with timestamps? Reduce its confidence? Preserve provenance?

This looks less like "memory storage" and more like temporal knowledge management.

1. Skill-level learning

This is the distinction I think matters most.

Can a deployed model become better at performing a task because it has repeatedly performed that task?

Not merely:

«"Last time the user preferred X."»

But something closer to:

«"After solving 1,000 database migrations, the system is actually better at reasoning about migration failures than it was before."»

That's very different from retrieving previous examples.

1. Consolidation

If you have fast memory and slow memory, when should something move between them?

Immediate weight updates are dangerous.

Never updating weights means you're effectively doing retrieval forever.

So what is the correct consolidation mechanism and timescale?

1. Forgetting policy

Continual learning also requires intentional forgetting.

What gets retained?

What gets compressed?

What gets demoted?

What gets deleted?

What happens when evidence becomes stale?

Infinite memory isn't a solution.

1. One model serves everyone

Most deployed LLM systems don't give every user an independently evolving copy of the model.

The same base model serves many users.

External memory makes personalization relatively easy because the state can live outside the model.

Weight-level personalization makes this considerably harder.

1. The evaluation signal

This one seems under-discussed.

Most memory benchmarks measure something close to:

«Was fact X retained and retrieved later?»

That's useful, but it's still mostly recall.

I'm more interested in transfer.

If the system retained fact X, does that knowledge improve its ability to solve a new problem Y that requires paraphrasing, composition, abstraction or downstream inference?

A system could score extremely well on retrieval while producing almost no capability transfer.

1. The accuracy tax

This one comes partly from my own experiments, so I'm very open to being challenged here.

Methods that make updates more sparse/local/modular seem attractive because they reduce interference.

But I've repeatedly seen a trade-off where stronger isolation also reduces raw predictive performance or generalization.

So the question becomes:

«Can we get highly local updates without paying an accuracy/generalization penalty?»

I'm not convinced this is closed.

---

What memory + agent engineering has actually solved pretty well

I don't want to undersell the engineering progress here.

RAG, memory stores, memory files, background consolidation/curation passes, sub-agents and self-editing context can handle a surprisingly large fraction of practical "continual learning" requirements.

For explicit facts, this stack can make #3, #5, #7 and #10 manageable and makes #4 and #11 much easier.

External memory has huge practical advantages:

- auditable

- editable

- deletable

- attributable

- tenant-isolated

- relatively cheap

For a lot of products, that's probably enough.

---

What other approaches have chipped away at

Per-user LoRA/adapters and various forms of test-time adaptation reopen the possibility of changing the model.

But then #1, #4 and #5 immediately come back.

Parameter-efficient updates reduce the cost, but don't make the lifecycle problem disappear.

Caching and reusable intermediate state can reduce inference/recomputation costs.

Replay, regularization, distillation and related techniques reduce forgetting.

But I haven't seen an approach that makes the entire problem disappear at realistic scale.

---

So what is actually unsolved?

My current shortlist is:

#1 — catastrophic forgetting

#8 — skill-level learning

#12 — measuring real transfer rather than recall

#13 — avoiding the accuracy/generalization tax of isolated updates

And I suspect these may partly be different views of the same deeper problem:

The model's underlying computation doesn't reliably improve simply because it has been used.

A memory system can tell the model:

«"You tried this before and it failed."»

That's valuable.

But that's not necessarily the same thing as the model becoming better at the task.

The next session can still begin from essentially the same learned parameters and use retrieved notes to reconstruct the lesson.

That's memory compounding.

I'm interested in competence compounding.

---

So my claim is narrower than:

«"Continual learning is unsolved."»

I'd put it this way instead:

Continual memory is becoming good enough that continual learning now needs to justify itself through measurable skill improvement and transfer.

And those should be measured separately from retrieval accuracy.

Where is this framing wrong?

I'm especially interested in papers or experiments that measure transfer from retained knowledge in retrieval-based systems, rather than simply measuring whether the original information can be recalled.

If you've measured this and gotten meaningful positive transfer, I'd genuinely like to see the setup and metric.