AI alignment
Sources are over 18 months old.
Publication dates and source age
Sources counted: 4
Newest dated source: 2022-12-15
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Open in the glossary Reading notes · Structured record
In plain language
Work on making AI behavior fit intended goals, constraints and human judgments. [1] [2]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Limits & distinctions
Alignment is one part of AI safety. Following a user's wishes can still cause harm, and better scores on an alignment test do not settle every risk. [1] [4]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
A fuller explanation
Some alignment work trains models to follow instructions or avoid harmful responses, using human feedback or written principles. Broader research also asks whether a learned system pursues the objective its developers intended. [1] [2] [3]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
How it relates to the map
Ask whose goals are being followed, how conflicts are handled and what evidence supports the claim. [1]
Reference this explanation or suggest a correction
Link to this explanation · Suggest a correction · How corrections work
Share this page
https://theaiatlas.org/ideas/alignment/
Download a share image · Vector image
Image previews are summaries. Keep the page link so readers can check the evidence.
Sources and what we read
1. Training language models to follow instructions with human feedback
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2022-03-04
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read abstract, section 3.1's demonstrations/comparisons/reward-model procedure and section 5.3's limitations. Human preference judgments and improved results on the authors' tasks do not establish universal alignment or safety. Publication date checked on the arXiv abstract page.
2. Constitutional AI: Harmlessness from AI Feedback
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2022-12-15
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read abstract and metadata describing model critiques, revisions and AI preference feedback guided by human-written principles. Used as an example of alignment methods; the paper's claims do not certify every output as harmless.
3. Risks from Learned Optimization in Advanced Machine Learning Systems
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2019-06-05
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read abstract and revision history introducing mesa-optimization and the relation between learned and training objectives.
4. Concrete Problems in AI Safety
Source is over 18 months old.
Publication dates and source age
Sources counted: 1
Newest dated source: 2016-06-21
Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.
Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.
Read the abstract's five accident-risk problems including reward hacking and distributional shift. Does not give a general catastrophe probability.
Edition and machine-readable evidence
Content version 0.20.0. Evidence cutoff 2026-09-15; this does not mean every source was read on that day.
Pinned complete dataset · Complete evidence page · Agent consumption guide