Glossary
Eval
A structured test of what an AI model can do or how it behaves, including tests for risks.
"Eval" is industry shorthand for evaluation: any structured test of a model. Benchmarks are one kind. Others include safety evaluations that check whether a model can help with dangerous tasks, tests of whether it follows rules, and studies of how it behaves over long agent tasks.
Developers run evals before release and often summarize the results in a system card. Some governments and outside groups also run evals, sometimes before a model is public.
An eval only measures what it was designed to test, under the conditions used. Results can change with different prompts or tools, and a model that passes a safety eval has not been shown to be safe in every setting.
Read more: our explainer