Infrastructure
Temperature
Also known as: sampling temperature
A setting that controls how random or predictable an AI model's output is, low for focused and consistent, high for varied and creative.
Temperature is a dial that decides how adventurous an AI is with its wording. Turn it low and the AI plays it safe, giving focused, predictable answers, great for facts, code or anything that must be consistent. Turn it up and it takes more chances, producing more varied and creative results, handy for brainstorming or writing, but also more likely to wander off.
Temperature is a number, usually between 0 and about 1 (sometimes up to 2), that controls the randomness of a language model's output. At each step the model produces a probability for every possible next token; temperature reshapes that distribution before one is chosen.
What the values do
- Low (near 0): the model almost always picks the most likely token. Deterministic, focused, repetitive. Best for factual answers, extraction and code.
- Medium (~0.7): a balance of coherence and variety. A common default for chat.
- High (>1): flatter odds, more surprising choices. Good for creative variety, but raises the risk of incoherence and hallucination.
Temperature is often used alongside top-p (nucleus sampling), which limits choices to the most probable tokens.
Temperature T scales the logits before the softmax: p_i ∝ exp(z_i / T). As T → 0 the distribution collapses toward the argmax (greedy decoding); as T grows it flattens toward uniform, increasing entropy and the chance of low-probability tokens. It is one lever in a family of decoding controls, typically combined with nucleus sampling (top-p) and top-k truncation to cut the unreliable tail while preserving diversity (Holtzman et al. 2019).
Practical guidance: use low temperature for tasks requiring faithfulness, determinism or structured output, and higher temperature for ideation. Note that temperature does not change the model's knowledge or "confidence" in any calibrated sense, it only alters sampling; two runs at T>0 can differ purely by chance. For reproducibility, fix a low temperature (and seed, where supported). It interacts with prompt engineering: a well-constrained prompt plus low temperature is the reliable-output recipe.