Models
Foundation Model
Also known as: base model
A large model trained on broad data that serves as a versatile base, adaptable to many different downstream tasks.
A foundation model is a big, general-purpose AI trained on a huge amount of broad data, meant to be a starting point for many different uses. Rather than building a new AI from scratch for each job, you take one powerful foundation model and adapt it, to write, to code, to analyse images. The large language models behind today's chatbots are the best-known example.
A foundation model is a model trained on broad, large-scale data that can be adapted (via fine-tuning, prompting or retrieval) to a wide range of downstream tasks. The term, coined by Stanford researchers in 2021, captures a shift in how AI is built.
The shift it names
- Before: train a separate specialised model for each task, each needing its own labelled data.
- After: pretrain one large general model once, then adapt it cheaply to many tasks.
LLMs are foundation models for text; there are also foundation models for images, code, audio and multimodal data.
Why it matters
It concentrates capability (and cost, and risk) into a few base models that many applications build on, which is powerful but also raises concerns about reliability, bias and dependence on a handful of providers.
"Foundation model" names any model trained on broad data at scale, typically via self-supervised pretraining, that acquires general capabilities transferable to many downstream tasks through adaptation (Bommasani et al. 2021). The term emphasises two properties: emergence (capabilities not explicitly designed appear with scale) and homogenisation (many applications inherit from the same few bases, so their strengths, weaknesses and biases propagate).
Architecturally most are transformers; LLMs are the textual instance, alongside vision, code, audio and multimodal foundation models. The paradigm reorganises the ML lifecycle around pretrain-then-adapt, shifting cost to a large upfront training run and downstream effort to fine-tuning, in-context learning and retrieval. It also concentrates systemic risk, single points of failure, correlated errors, provenance and governance, motivating evaluation, transparency and safety work at the base-model layer. "Base model" specifically denotes the pre-alignment checkpoint before instruction tuning.