Training
Transfer Learning
Also known as: knowledge transfer
Reusing a model trained on one task as the starting point for another, so you need far less data and compute for the new task.
Transfer learning is reusing what a model already learned for a new job, instead of starting from scratch. A model that learned general vision from millions of images has already learned edges, shapes and textures; to teach it to spot a specific product defect, you only need to adjust it a little with a small set of examples. It saves enormous time, data and cost, and it is the reason a small team can build capable AI.
Transfer learning takes a model trained on one (usually large, general) task and adapts it to a different (usually smaller, specific) task. The general knowledge "transfers", so the new task needs much less data and training.
The modern pattern
This is exactly how foundation models are used: 1. Pretrain a big model once on broad data (expensive, done by a few). 2. Transfer it to countless specific tasks via fine-tuning, prompting or retrieval (cheap, done by many).
Why it matters
It democratised AI: you no longer need a massive dataset and compute budget to build something useful, you build on top of a pretrained model.
Transfer learning reuses representations learned on a source task/domain to improve learning on a target, reducing sample and compute requirements and enabling generalisation from limited target data (Pan & Yang 2010). In deep learning it is realised through feature reuse: lower layers learn broadly useful features that transfer, with adaptation via fine-tuning, feature extraction (frozen backbone), or parameter-efficient methods (LoRA/adapters).
It is the conceptual backbone of the pretrain-then-adapt foundation-model paradigm, and modern in-context learning can be seen as transfer without weight updates. Considerations include negative transfer (source-target mismatch hurting performance), domain shift, and catastrophic forgetting during adaptation. The broad transferability of large self-supervised models, one base serving many downstream tasks, is precisely what makes contemporary AI economically and practically scalable.