Latency
Publication dates and source age
Sources counted: 1
Newest dated source: 2025-09
At least one source was published within the 18-month window.
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Open in the glossary Reading notes · Structured record
In plain language
The waiting time for a system’s response. For generated text, the first visible word and the completed answer have different waiting times. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Limits & distinctions
Throughput describes how much work a system processes in a period. A system that handles many requests overall does not necessarily answer each individual request quickly. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
A fuller explanation
Latency is a duration. Language-model benchmarks can measure time until the first output token and the time taken to produce later tokens. These measures describe different parts of the waiting experience. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
How it relates to the map
Map context: response speed is a product-performance measure. It is separate from the map’s axis about the pace of developing more capable AI. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Share this page
https://theaiatlas.org/ideas/latency/
Download a share image · Vector image
Image previews are summaries. Keep the page link so readers can check the evidence.
Sources and what we read
1. MLPerf Inference 5.1: Benchmarking Small LLMs with Llama3.1-8B
Publication dates and source age
Sources counted: 1
Newest dated source: 2025-09
At least one source was published within the 18-month window.
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read the performance-metrics section distinguishing time to first token, subsequent token timing and throughput. Specific benchmark targets are not presented as universal user requirements.
Edition and machine-readable evidence
Content version 0.20.0. Evidence cutoff 2026-09-15; this does not mean every source was read on that day.
Pinned complete dataset · Complete evidence page · Agent consumption guide