A field guide to AI positions

Benchmark

Evaluation method.

Source is over 18 months old.

Publication dates and source age

Sources counted: 1

Newest dated source: 2022-11-16

Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.

Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.

Open in the glossary Reading notes · Structured record

Prepared with AI assistance. Not independently fact-checked. Original sources and qualifications are below.

In plain language

A shared test for comparing systems on chosen tasks; its score covers those tests, not every possible use. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-benchmark-1a5192ba3d7825ca26b24a72. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

Limits & distinctions

Evaluation is the broader investigation; a benchmark is one tool within it. A high score can leave relevant languages, users or failure types untested. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-benchmark-09422ce4d5c74cb753a9bb98. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

A fuller explanation

A benchmark combines tasks, scoring rules and a test setup. Using the same setup makes comparisons more informative. Accuracy, cost, bias and robustness can produce different pictures of the same model. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-benchmark-1e4c26398ee834b2e16dc7b8. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

How it relates to the map

Atlas reading question: which tasks and conditions support a public capability claim? [1]

Reference this explanation or suggest a correction

Editorial interpretation or qualification. Claim ID: claim-term-benchmark-25f52a21ad9f2e54a62ed1b2. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

Share this page

https://theaiatlas.org/ideas/benchmark/

Download a share image · Vector image

Image previews are summaries. Keep the page link so readers can check the evidence.

Sources and what we read

  1. 1. Holistic Evaluation of Language Models

    Percy Liang and coauthors / Stanford CRFM, arXiv · First-hand source · Published 2022-11-16 · Updated 2023-10-01 · Material last read 2026-09-15

    Source is over 18 months old.

    Publication dates and source age

    Sources counted: 1

    Newest dated source: 2022-11-16

    Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.

    Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.

    Read scope is described in the source note.

    No archive check recorded.

    Read the abstract and version history, including multiple use cases and metrics, standardized comparisons and acknowledged coverage gaps. Used for evaluation principles; historical model scores are not presented as current rankings.

    Source ID: helm-paper

Edition and machine-readable evidence

Content version 0.20.0. Evidence cutoff 2026-09-15; this does not mean every source was read on that day.

Pinned complete dataset · Complete evidence page · Agent consumption guide