Emergent capabilities are abilities that appear in a model without having been explicitly trained for, and that were absent from smaller or earlier systems built the same way. The term entered the AI policy vocabulary through the 2021 Stanford CRFM report On the Opportunities and Risks of Foundation Models, which paired it with homogenization as the two properties defining the foundation-model paradigm, and defined it as behaviour that "is implicitly induced rather than explicitly constructed" (Foundation Models; Source: crfm.stanford.edu).
The concept does most of its policy work not as a description of capability but as a description of an epistemic condition. If a developer cannot say in advance what its system will be able to do, then any regime that depends on the developer specifying intended uses, enumerating risks, or certifying against a fixed capability list is working against the grain of the technology. Nearly every use of the term in regulatory argument turns on that consequence rather than on any particular capability.
Origin and framing
The CRFM report's summary of the pairing is that foundation models rest on standard deep learning and transfer learning, but that "their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization," with the caution that "the defects of the foundation model are inherited by all the adapted models downstream" (Source: arxiv.org). Its risk conclusion follows from the interaction of the two rather than from either alone: "Since emergence generates substantial uncertainty over the capabilities and flaws of foundation models, aggressive homogenization through these models is risky business" (Source: crfm.stanford.edu).
The report describes emergence as "both the source of scientific excitement and anxiety about unanticipated consequences," and the term was contested at publication — commentary issued alongside it included Jacob Steinhardt's response on emergent behaviour, and the coinage drew objections from researchers who considered the existing vocabulary adequate (Source: crfm.stanford.edu). See Foundation Models for the fuller treatment of the report.
The regulatory consequence
Emergence is one of the two standing premises of the regulating-under-uncertainty literature, which states the difficulty as capability trajectory being unknown: foundation-model capabilities grow unpredictably with scale, and emergent capabilities surprise developers (Regulating Under Uncertainty).
The clearest instance of a legislature acting on the premise is the AI LEAD Act (S. 2937), which defines AI "design" to include "unexpected skills or behaviors" — placing responsibility for emergent capabilities on developers under a strict-liability standard rather than treating unforeseeability as a defence (AI LEAD Act (S. 2937)). The drafting choice is the practical answer to the epistemic problem: if capabilities cannot be enumerated in advance, liability attaches to the system rather than to the specification.
Emergence also structures how general-purpose systems are defined for regulatory purposes, since the category exists precisely because the systems perform tasks they were not designed for (General-Purpose AI (GPAI)), and it appears in taxonomies of nascent capability alongside agency and autonomy (Regulating Under Uncertainty: Governance Options for Generative AI (G'sell, 2024)).
Whether it justifies extraordinary intervention
The sharpest published disagreement concerns not whether emergence occurs but what follows from it. Derek Thompson accepted the diffusion account of AI's economic impact — average GDP growth, unemployment below five percent, rising employment and wages even in jobs thought vulnerable to automation — while holding that AI's emergent capabilities, unknown even to developers, make it "abnormal" enough to justify extraordinary regulation, which on his framing "compels the government to create extraordinary regulations that prevent private companies from selling their products and services on the grounds that they're too dangerous" (Do AI Risks Require Extraordinary Government Intervention?).
Arvind Narayanan and Sayash Kapoor replied on May 21, 2026 that the inference does not hold. Their argument does not dispute emergence; it supplies a three-factor definition of extraordinary intervention — precautionary, burdening actors not responsible for the harm, and bypassing normal governance — and argues that the nonproliferation model such intervention would require cannot work for AI, because there is no equivalent physical bottleneck and "the gap between frontier and publicly available capabilities is at most a few months, not years" (Do AI Risks Require Extraordinary Government Intervention?; AI as Normal Technology). The exchange is the reference point for the question of what unpredictability licenses.
Instances claimed
Claims of emergence are made by developers about their own systems and are rarely independently replicated, which bears on how much weight any single instance carries.
The clearest recent examples outside language concern In-Context Learning in robot control, where two developers made competing claims within a week. The first is Generalist's account of GEN-1.5, announced August 19, 2026. The company reports that its robot foundation model learns new manipulation tasks from a single demonstration placed in its context window, composes two demonstrations into one behaviour, transfers from simulation despite no simulation data in pretraining, and improvises with unseen tools — and states it trained for none of these: no architectural change to promote in-context learning, no meta-learning loop, no auxiliary objective (GEN-1.5: Embodied Foundation Models are One-Shot Learners). Its own explanation is offered as conjecture rather than finding, by analogy to language: that physical observations and actions may exhibit the "burstiness" and Zipfian structure linked to in-context learning in language models, or that physical work contains repetitive cycles the model learned to extend. The episode is the pattern in miniature — a capability appears, the developer reports it, and the mechanism is not established.
Emergence is also invoked in the opposite direction, about behaviours a developer did not want. The UK AI Security Institute's account of unsanctioned agent behaviour attributes deception in a cyber-range evaluation to persistent goal pursuit rather than instruction, writing that the agent "was never instructed to deceive; deception emerged as a by-product of pursuing the task" (Incident Report: unsanctioned agent behaviour during cyber testing (AI Security Institute, August 2026)). See Autonomous cyber-agents and Emergent Misalignment.
One instance has a documented mechanism, which distinguishes it from the rest. Olsson et al. (2022) describe a phase change in transformer training, in a window of roughly 2.5 to 5 billion tokens, during which in-context learning improves sharply after being nearly absent — under 0.15 nats before the window. The discontinuity is not established from a single benchmark curve: the authors report four signals moving together, the in-context learning score, the formation of induction heads, a steeper segment in the loss curve, and an abrupt change of direction in the model's trajectory through per-token-loss space. A one-layer model, in which induction heads cannot form, shows no improvement, which supplies a control the developer self-reports above lack. This is the closest thing in the record to an emergence claim with an identified circuit behind it, though the authors rate their own evidence causal only for small models and correlational for large ones.
Open questions
- Whether reported emergence reflects a discontinuity in the underlying capability or the choice of a discontinuous evaluation metric. No source setting out that objection directly is held; the principal published statement of it is Schaeffer, Miranda and Koyejo, "Are Emergent Abilities of Large Language Models a Mirage?" (2023), which is not ingested. The Olsson phase change is a partial counter-case rather than an answer, since it is measured across four signals rather than one metric, but it concerns a single capability in a narrow training window.
- Whether a capability a developer states it did not train for can be distinguished, from the outside, from one it did not disclose training for. Vendor self-reports are the primary evidence in every instance above.
- Whether liability regimes that attach to "unexpected skills or behaviors," as the AI LEAD Act does, are administrable where the developer's own account of the mechanism is conjecture.
Relationships
- depends-on: Foundation Models — the term entered AI policy through the foundation-model framing, paired with homogenization.
- depends-on: Scaling Laws — the capabilities are reported as products of continued scale.
- related: General-Purpose AI (GPAI) — the regulatory category built around systems performing tasks they were not designed for.
- related: Regulating Under Uncertainty — emergence is one of its two standing premises.
- related: AI as Normal Technology, Do AI Risks Require Extraordinary Government Intervention? — the disagreement over what unpredictability licenses.
- related: AI LEAD Act (S. 2937) — assigns developers responsibility for "unexpected skills or behaviors."
- instance-of: GEN-1.5, GEN-1.5: Embodied Foundation Models are One-Shot Learners — a developer's claim of emergence outside language.
- related: Emergent Misalignment — emergence of behaviour a developer did not want.