Infrastructure
TPUTPU
Also known as: Tensor Processing Unit
A chip designed by Google specifically for AI workloads, an alternative to GPUs for training and running large models.
A TPU is a special AI chip built by Google, made specifically to run the maths that AI needs, rather than being adapted from graphics chips like GPUs. It is another type of "engine" for training and running AI models. For most people the distinction does not matter, but it is part of why some AI runs on Google's systems and other AI on GPU-based systems.
A Tensor Processing Unit (TPU) is a custom AI accelerator (an ASIC) designed by Google for machine-learning workloads. It plays a similar role to a GPU: performing the massive parallel matrix maths of deep learning very fast, both for training and inference.
GPU vs TPU
- GPU: originally for graphics, extremely versatile, from many vendors, the industry default.
- TPU: purpose-built for AI, tightly integrated with Google's software and cloud.
Both target the same job. Which is used depends mostly on the provider's infrastructure. TPUs are a leading example of custom AI silicon, a category that also includes chips from other companies.
TPUs are Google-designed ASICs optimised for dense linear algebra, built around large systolic-array matrix units with high-bandwidth memory and fast inter-chip interconnect for pod-scale compute (Jouppi et al. 2017). Versus general-purpose GPUs, they trade flexibility for efficiency on the matmul-dominated workloads of deep learning, and are co-designed with a software stack (XLA compilation).
They exemplify the broader shift to domain-specific AI accelerators (alongside other custom ASICs and inference chips), driven by the end of general CPU scaling and the economics of large-model training/serving. As with all accelerators, memory capacity/bandwidth and interconnect, not just peak FLOPs, often bound real performance, and effective use depends on parallelism strategy and compiler support. TPU availability is part of the strategic compute landscape that shapes who can train frontier models, and a differentiator for Google's cloud and model programmes.