Models
Small Language ModelSLM
Also known as: compact model
A language model with relatively few parameters, small enough to run cheaply, on-device or offline, while still being capable for many tasks.
A small language model is a compact version of the AI behind chatbots, small enough to run on a laptop or even a phone, without a giant data centre. It is not as broadly capable as the biggest models, but for many everyday tasks it is fast, cheap, private (your data can stay on your device) and works offline. The gap with the giants has been shrinking fast.
A small language model (SLM) is an LLM with a relatively small parameter count, chosen for efficiency. What counts as "small" shifts over time, but the point is a model light enough to run at low cost, on modest hardware, or on-device.
Why they matter
- Cost and speed: far cheaper and faster at inference.
- Privacy: can run locally, so sensitive data need not leave the device.
- Offline and edge use: works without a connection.
How they got good
Better training data, distillation from larger models, and quantization let small models punch above their weight, so a well-trained SLM can beat a much larger but older model on many tasks.
"Small language model" is a relative, moving designation for parameter-efficient LLMs optimised for low-cost, low-latency or on-device inference. Recent results show carefully curated, high-quality (often synthetic or distilled) training data lets compact models rival far larger predecessors on many benchmarks (e.g. Phi-3; Abdin et al. 2024), reflecting that data quality and training method, not size alone, drive capability.
SLMs are typically deployed with quantization and efficient serving, and increasingly as specialised components, routed to by a larger system, embedded in agents, or fine-tuned per task, rather than as general do-everything assistants. Trade-offs versus frontier LLMs: weaker broad knowledge, long-context handling and complex reasoning, against major gains in cost, latency, privacy and controllability. The economics favour "right-sizing", using the smallest model that meets a task's quality bar, which makes SLMs central to practical, cost-aware AI deployment.