Infrastructure
Compute
Also known as: computing power, FLOPs
The raw amount of processing power used to train or run an AI model, one of the main factors deciding how capable it can be.
"Compute" is shorthand for the amount of computing power poured into an AI, how many calculations, on how many chips, for how long. It is one of the three big ingredients of a capable model, alongside data and clever design. Training a frontier model can take thousands of specialised chips running for weeks, which is why serious AI is expensive and why access to compute is such a talked-about resource.
Compute refers to the processing power consumed by an AI workload, usually measured in operations (FLOPs) or in chip-time (e.g. "GPU-hours"). It shows up in two places: the large one-off cost of training a model, and the ongoing cost of inference (running it for users).
Why it is a headline topic
- Capability: more training compute, spent well, tends to yield more capable models.
- Cost and access: frontier training needs huge clusters of GPUs, so compute is a real economic and strategic bottleneck.
- Efficiency: a lot of AI engineering is about getting more capability per unit of compute.
Compute, data and model size (parameters) are linked by scaling laws, empirical patterns describing how performance improves as you scale them together.
Compute quantifies the arithmetic work of a workload, commonly in FLOPs for training budgets and in device-time for planning; a useful heuristic for dense transformer training is C ≈ 6ND (N parameters, D training tokens). It is one axis of the empirical scaling laws relating loss to model size, data and compute (Kaplan et al. 2020), later refined toward compute-optimal parameter/data balance. Training compute for frontier systems has grown by orders of magnitude, making cluster scale, interconnect, power and cost first-order determinants of who can build them.
Two shifts matter. First, compute-optimality reframed "spend on a bigger model" as "balance model and data for a fixed budget". Second, test-time compute, allocating more inference to a single query (long reasoning traces, sampling/search), buys accuracy without larger weights, moving part of the compute burden from training to serving. Efficiency techniques (quantisation, sparsity/mixture-of-experts, better kernels and parallelism) target more capability per FLOP, while GPU supply remains a strategic constraint on the field.