Training & pretraining
Dated sources are over 18 months old; other dates are unknown.
Publication dates and source age
Sources counted: 5
Newest dated source: 2024-07
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Some publication dates are unknown; the newest dated source may not be the newest source overall.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Open in the glossary Reading notes · Structured record
In plain language
The process that adjusts a model's learned settings using data and feedback. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Limits & distinctions
A lower training error concerns the chosen objective and examples. Testing on new, relevant tasks is needed to assess how useful the model is elsewhere. [4] [5]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
A fuller explanation
During neural-network training, software compares predictions with a training objective and adjusts parameters to reduce error. Pretraining is the initial broad training stage; later training can adapt the model to particular tasks. [1] [2]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
How it relates to the map
This helps identify whether a proposal concerns developing a model or using an existing one. [1] [3]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Share this page
https://theaiatlas.org/ideas/training/
Download a share image · Vector image
Image previews are summaries. Keep the page link so readers can check the evidence.
Sources and what we read
1. Linear regression: Gradient descent
Publication dates and source age
Sources counted: 1
Publication dates are unavailable.
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Some publication dates are unknown; the newest dated source may not be the newest source overall.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read the iterative prediction, loss and parameter-update explanation. The page's guarantees for convex linear regression are not extended here to neural-network training. Updated date follows the page; initial publication is unspecified.
2. LLMs: Fine-tuning, distillation, and prompt engineering
Publication dates and source age
Sources counted: 1
Publication dates are unavailable.
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Some publication dates are unknown; the newest dated source may not be the newest source overall.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read fine-tuning and prompt engineering, including the distinction between parameter updates and examples supplied as input. Used to distinguish these processes, without adopting general claims that fine-tuning is always necessary or improves every task.
3. Machine Learning Glossary
Publication dates and source age
Sources counted: 1
Publication dates are unavailable.
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Some publication dates are unknown; the newest dated source may not be the newest source overall.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read the artificial intelligence, deep model, context window, inference and chat entries. Used for terminology, not product performance claims. Updated date follows the earlier displayed page date; initial publication is unspecified. The chat entry was reread during the same-day beginner-content review. Also read the generalization and compute entries.
4. Holistic Evaluation of Language Models
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2022-11-16
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read the abstract and version history, including multiple use cases and metrics, standardized comparisons and acknowledged coverage gaps. Used for evaluation principles; historical model scores are not presented as current rankings.
5. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2024-07
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read introduction, section 2.2 on confabulation, and selected MEASURE actions 2.3, 2.5, 2.6, 2.7 and 2.9 concerning evaluation evidence, generalization, citations, generated-code review and safeguards. A voluntary risk-management profile; no claim that all 64 pages or every referenced study was reviewed.
Edition and machine-readable evidence
Content version 0.20.0. Evidence cutoff 2026-09-15; this does not mean every source was read on that day.
Pinned complete dataset · Complete evidence page · Agent consumption guide