Glossary
Benchmark
A standard test used to measure and compare AI models on a set task.
A benchmark is a fixed set of questions or tasks, with a scoring method, used to compare AI systems. Examples range from math problems and coding tasks to medical exam questions and simulated office work.
Benchmarks let companies and researchers track progress, but scores depend on how the test was run: the prompts used, the number of attempts allowed, whether tools were available, and who ran it. A company's own results are claims until others reproduce them.
A high score shows a model does well on that test, not that it will do well at the real job the test is meant to represent. Watch for contamination and saturation.
Read more: our explainer
See also
In our coverage
- GSA's AI contract clause takes effect, barring vendors from training models on government data
- Google announces Gemini 4 Argon, first limiting access to vetted cyber defenders
- OpenAI releases GPT-6.1 Sol, says it nearly matches GPT-6 Astra at about a fifth of the price
- How to read an AI benchmark claim
- AI for science: telling a new capability from a promising demo