Models
Parameter
Also known as: weights
One of the many internal, adjustable numbers a model tunes during training; their count (e.g. "70 billion parameters") roughly indicates a model's size.
Parameters are the millions or billions of little adjustable numbers inside an AI model. During training the model tweaks them again and again until it gets good at its task; together they store everything it has "learned". When you hear a model has "70 billion parameters", that is a rough measure of how big and capable it is, a bit like engine size for a car.
A parameter is a single learned value inside a neural network, mostly the connection strengths called weights (plus biases). Training adjusts all of them so the model's outputs improve. The total parameter count is the most common headline measure of a model's size.
What the count tells you (and doesn't)
- More parameters generally means more capacity to learn, but also more compute and memory to train and run.
- Bigger is not automatically better: data quality, training method and architecture matter enormously. A well-trained smaller model can beat a poorly-trained larger one.
Parameters are fixed after training; adapting them to a new task is fine-tuning, whereas prompting changes behaviour without touching them.
Parameters are the trainable variables of a model, principally weight matrices and biases, optimised by gradient descent to minimise the training loss; they constitute the model's parametric memory. Count is a coarse capacity proxy that trades off against compute and serving cost, and it interacts with data via scaling laws: the compute-optimal analysis (Hoffmann et al. 2022) showed many large models were undertrained, i.e. parameters and training tokens should scale together, reframing "bigger model" as "better-balanced model".
Distinguish parameters from hyperparameters (configuration set before training: learning rate, batch size, layer count) and from activations (transient values at inference). Parameter-efficient fine-tuning (LoRA/QLoRA) adapts a model by training a small number of additional parameters while freezing the rest. Note that reported counts can mislead across architectures, e.g. mixture-of-experts models have a large total parameter count but activate only a subset per token, so "active parameters" better reflects per-inference compute.