Training

Catastrophic Forgetting

Also known as: catastrophic interference

When training a model on something new makes it forget things it previously knew; a key challenge in updating AI models.

Choose your level

Catastrophic forgetting is when teaching an AI something new causes it to forget what it already knew, a bit like cramming for one exam and wiping out last month's material. It is a real headache when you want to update or specialise a model without losing its general abilities, and it is one reason keeping AI current is harder than it sounds.

Catastrophic forgetting happens when fine-tuning a neural network on a new task overwrites the weights that stored earlier knowledge, so performance on old tasks drops sharply. Neural networks do not naturally add knowledge without disturbing what is already there.

Why it matters

Specialising a model can degrade its general skills.
Continual learning, updating a model over time, is hard because of it.

How it is managed

Parameter-efficient tuning like LoRA: freeze the base model, train small add-ons, so the original knowledge is preserved.
Mixing old and new data during training.
Regularisation that protects important weights.
Often it is easier to use retrieval for new knowledge instead of retraining.

Catastrophic forgetting is the sharp loss of previously acquired capabilities when a network is trained sequentially on new data, arising because gradient updates on a new distribution overwrite weights encoding prior tasks, a central obstacle to continual/lifelong learning. Mitigations include regularisation that penalises changes to parameters important for old tasks (EWC; Kirkpatrick et al. 2017), rehearsal/replay of prior data, and architectural isolation.

In the LLM era it manifests when fine-tuning degrades general or safety-relevant capabilities. Practical countermeasures favour parameter-efficient methods (LoRA/adapters that freeze the base), data mixing (replaying pretraining/instruction data), conservative learning rates, and, crucially, preferring non-parametric knowledge injection (RAG/in-context) over weight updates when the goal is adding facts rather than changing behaviour. It frames a core design heuristic: retrieval for updatable knowledge, careful fine-tuning for durable behaviour, since naive retraining trades new competence for old.

← Back to the glossary