Foundations
Supervised Learning
Also known as: learning from labelled data
Training a model on examples that come with the correct answers, so it learns to predict those answers on new data.
Supervised learning is teaching a computer with a labelled answer key. Show it thousands of emails already marked "spam" or "not spam", and it learns to sort new emails on its own. The word "supervised" just means every training example came with the right answer attached, so the model has something to check itself against while learning.
Supervised learning is the most common form of machine learning. The model is trained on labelled examples, each input paired with its correct output, and learns a mapping from input to output that it can apply to new, unseen data.
Two main types
- Classification: predict a category (spam / not spam, which disease, which digit).
- Regression: predict a number (house price, tomorrow's temperature).
The catch
It needs labelled data, and labelling is often expensive and slow. That is why other approaches matter: unsupervised learning needs no labels, and self-supervised learning (the trick behind LLMs) creates labels automatically from the data itself.
Supervised learning fits a function f: X → Y from labelled pairs {(xᵢ, yᵢ)} by minimising an empirical loss, aiming to generalise to the underlying distribution rather than memorise the sample (Hastie et al. 2009). The central tension is the bias-variance trade-off and the train/test generalisation gap, managed through model capacity, regularisation, cross-validation and sufficient, representative data.
It contrasts with unsupervised learning (structure without labels) and reinforcement learning (reward-driven). Crucially, the LLM pipeline uses supervised signals at several points: self-supervised pretraining derives labels from the data (next-token prediction), and later supervised fine-tuning on curated demonstrations shapes instruction-following. Practical failure modes trace to label noise, distribution shift between training and deployment, and label bias, which the model faithfully reproduces.