A field guide to AI positions

Mixture of experts (MoE)

Inside a model.

Source is over 18 months old.

Publication dates and source age

Sources counted: 1

Newest dated source: 2024-01-08

Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.

Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.

Open in the glossary Reading notes · Structured record

Prepared with AI assistance. Not independently fact-checked. Original sources and qualifications are below.

In plain language

A model design with several calculation blocks called experts; some versions activate only a subset for each input. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-mixture-of-experts-1a5192ba3d7825ca26b24a72. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

Limits & distinctions

Experts are numerical components, not independent professionals. The name alone does not show that each block has a clear subject specialty. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-mixture-of-experts-09422ce4d5c74cb753a9bb98. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

A fuller explanation

In sparse MoE models such as Mixtral, a routing component selects only some expert blocks for each text piece. This allows more total parameters than are active for that piece. [1]

Reference this explanation or suggest a correction

Explanation based on the linked sources. Claim ID: claim-term-mixture-of-experts-1e4c26398ee834b2e16dc7b8. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

How it relates to the map

Distinguish total from active parameter counts when reading model-size comparisons. [1]

Reference this explanation or suggest a correction

Editorial interpretation or qualification. Claim ID: claim-term-mixture-of-experts-25f52a21ad9f2e54a62ed1b2. Edition 0.20.0.

Link to this explanation · Suggest a correction · How corrections work

Share this page

https://theaiatlas.org/ideas/mixture-of-experts/

Download a share image · Vector image

Image previews are summaries. Keep the page link so readers can check the evidence.

Sources and what we read

  1. 1. Mixtral of Experts

    Albert Q. Jiang and coauthors / arXiv · First-hand source · Published 2024-01-08 · Material last read 2026-09-15

    Source is over 18 months old.

    Publication dates and source age

    Sources counted: 1

    Newest dated source: 2024-01-08

    Assessed at this edition's evidence cutoff: 2026-09-15. 18-month boundary: 2025-03-15.

    Publication age does not tell us whether a claim is still valid. Reading an old source again does not make its publication date newer. An update date does not establish that the passage we used was updated.

    Read scope is described in the source note.

    No archive check recorded.

    Read the abstract and introduction describing token routing, expert blocks and active parameters. Publication date checked on arXiv. Mixtral illustrates sparse routing; its design is not assigned to every MoE. Also read section 5, Routing analysis, on whether routing follows subject areas.

    Source ID: glossary-model-mixture-of-experts

Edition and machine-readable evidence

Content version 0.20.0. Evidence cutoff 2026-09-15; this does not mean every source was read on that day.

Pinned complete dataset · Complete evidence page · Agent consumption guide