Prompting
In-context LearningICL
Also known as: learning from the prompt
A model's ability to pick up a new task from information in the prompt alone, without any change to its underlying training.
In-context learning is the surprising ability of a large AI to learn a task just from what you put in your message, on the fly, without being retrained. Show it the pattern once or twice and it continues correctly. It is why you can invent a little task on the spot ("label these as urgent or not") and the AI simply follows along.
In-context learning (ICL) is the phenomenon where a language model adapts its behaviour based purely on the prompt, instructions and any examples, without updating its internal weights. It is what makes both zero-shot (instruction only) and few-shot (with examples) prompting work.
Why it is a big deal
Before large models, teaching a system a new task meant collecting data and training. ICL means one general model can do thousands of tasks, each specified in plain language at the moment you ask. This is the foundation of prompt engineering.
Its limits
The "learning" lasts only for that request, nothing is remembered afterwards, and it is bounded by the context window. For durable new behaviour or knowledge, you need fine-tuning or retrieval.
In-context learning is the capacity of a pretrained autoregressive model to infer and execute a task from the conditioning context alone, with no gradient updates, first characterised at scale with GPT-3 (Brown et al. 2020). Empirically it improves with model scale and with the number and quality of demonstrations, and is sensitive to example ordering, formatting and label distribution.
Its mechanism is an active research area: proposals include the prompt inducing an implicit task inference over a latent skill acquired in pretraining, and analyses suggesting attention can implement optimisation-like updates within the forward pass (in-context "meta-learning"). Practically, ICL is ephemeral (no persistence across calls) and context-bounded, which frames the design choice: use ICL/prompting and retrieval for flexible, updatable behaviour, and fine-tuning when a change must be durable and weight-resident.