Infrastructure
GPUGPU
Also known as: graphics processing unit
A chip originally built for graphics that turns out to be ideal for AI, because it can do enormous numbers of calculations in parallel.
A GPU is a computer chip first designed to draw video-game graphics, which turned out to be perfect for AI. The reason: both graphics and AI need to do a huge number of simple sums at the same time. A regular processor does a few things very fast one after another; a GPU does thousands of things at once. That parallel power is what makes training and running modern AI possible.
A GPU (graphics processing unit) is a highly parallel processor. Where a CPU has a few powerful cores optimised for sequential tasks, a GPU has thousands of smaller cores that perform many calculations simultaneously, ideal for the matrix maths at the heart of deep learning.
Why GPUs matter for AI
- Training: adjusting billions of parameters over huge datasets needs massive parallel computation.
- Inference: running models for users, especially large ones, is far faster on GPUs.
- Scale: frontier models are trained on large clusters of thousands of GPUs working together.
The 2012 breakthrough that kicked off the deep-learning era was, in part, made possible by training a network on GPUs. Specialised AI chips such as TPUs serve similar roles.
GPUs are throughput-oriented, massively parallel processors whose SIMT architecture and high memory bandwidth suit the dense linear algebra (matrix multiplications) dominating neural-network training and inference. The modern deep-learning era is often dated to GPU-trained AlexNet (2012), and hardware scaling has since been a primary driver of capability alongside data and algorithms. Contemporary AI accelerators add tensor cores for mixed-precision matmul and high-bandwidth memory (HBM), with device-to-device interconnects enabling multi-GPU and multi-node parallelism.
The binding constraint is frequently memory bandwidth and capacity rather than raw FLOPs, particularly for autoregressive decode, which is memory-bound; hence techniques like quantisation, KV-cache management and fused kernels (e.g. flash attention). At the frontier, training runs span thousands of accelerators using tensor, pipeline and data parallelism, making interconnect, power and cluster economics first-order concerns, and GPU supply a strategic factor in who can build large models. Alternatives include TPUs and other custom ASICs targeting the same workloads.