Foundations
Machine LearningML
Also known as: statistical learning
A way of building software that learns patterns from examples instead of being programmed with explicit rules.
Machine learning is how a computer gets better at a task by looking at lots of examples, instead of a person writing down every rule by hand. Show it thousands of photos labelled "cat" or "not a cat", and it gradually learns to spot cats on its own, even in pictures it has never seen. Almost everything people call "AI" today works this way underneath.
Machine learning (ML) is the approach behind most modern artificial intelligence. Instead of a programmer specifying rules, the system is shown data and learns the patterns that connect inputs to outputs. A useful classic definition: a program learns if its performance on a task improves with experience.
Three main flavours
- Supervised learning: learn from labelled examples (photo → "cat"). Most practical business ML.
- Unsupervised learning: find structure in unlabelled data (grouping similar customers).
- Reinforcement learning: learn by trial and error, guided by rewards (game-playing, robotics).
Deep learning
The current wave is powered by deep learning: neural networks with many layers that learn their own useful features from raw data. This is what made large language models and image generators possible.
Machine learning is the discipline of fitting parameterised functions to data so as to minimise a loss over a task distribution, typically via gradient-based optimisation. The dominant modern subfield, deep learning, uses multi-layer neural networks whose representations are learned end-to-end rather than hand-engineered (Goodfellow, Bengio & Courville).
The canonical taxonomy is supervised, unsupervised and reinforcement learning, with important adjacent regimes: self-supervised learning, where labels are derived from the data itself, is what makes pretraining of LLMs scalable, since next-token prediction turns raw text into a supervision signal without human annotation. Core concerns are generalisation (the gap between training and unseen data), the bias-variance trade-off, regularisation, and the role of data quantity and quality, relationships partly formalised by empirical scaling laws. Downstream adaptation of a pretrained model to a specific task is handled by fine-tuning or in-context methods.