Training
Benchmark
Also known as: evaluation, eval
A standard test used to measure and compare how well AI models perform on specific tasks.
A benchmark is a standard exam for AI, a fixed set of questions or tasks used to score models and compare them fairly. When you hear "this model scored X% on a maths test" or "beats the previous model on coding", that is a benchmark. They are useful for tracking progress, but, like any exam, they can be gamed and do not capture everything that matters in real use.
A benchmark is a standardised dataset and scoring method for evaluating models on a task, general knowledge, maths, coding, reasoning, safety, and so on. Benchmarks let the field measure progress and compare systems on common ground.
Why they must be read carefully
- Contamination: if benchmark questions leaked into training data, high scores may reflect memorisation, not skill (a form of overfitting).
- Saturation: once models ace a benchmark, it stops distinguishing them, so new, harder ones are constantly created.
- Validity gap: doing well on a test does not guarantee real-world usefulness or reliability (e.g. it can still hallucinate).
Good evaluation combines multiple benchmarks with real-task testing and human judgement.
Benchmarks operationalise capability measurement via curated datasets and metrics (accuracy, pass@k, win-rate, calibration), spanning knowledge/reasoning (e.g. MMLU; Hendrycks et al. 2020), code (HumanEval), maths, agentic tasks and safety. They are indispensable for comparison yet fraught: contamination (test leakage into pretraining) inflates scores (overfitting to the benchmark), saturation erodes discriminative power, and construct validity is often weak, benchmark performance can diverge from deployment reliability and from hallucination/robustness behaviour.
Methodological rigour requires decontamination, held-out and dynamic/private test sets, prompt-sensitivity and multiple-seed reporting, and human or LLM-judge evaluations for open-ended tasks (each with its own biases). Increasingly the field favours capability-specific and adversarial evaluations, agentic/real-task benchmarks, and preference-based arenas over single aggregate scores. Benchmarks steer research and marketing alike, so Goodhart effects, optimising the metric rather than the underlying capability, are a persistent risk to interpret against.