In-context learning (ICL) is the acquisition of a new task by a pre-trained model from examples supplied in its input, with no gradient updates and no change to the model's weights. The examples occupy the model's context window; the adaptation lasts only as long as they remain there. Outside technical writing the same capability is usually called prompting.
The distinction it draws is between two ways of teaching a model something new. Under the fine-tuning paradigm, each application requires fresh data collection and a training run that alters the weights. Under in-context learning, a user introduces a novel concept entirely through the input and the model responds appropriately without any weight change (Introducing S1: In-Context Learning for Robotics). Which of the two a system relies on determines the marginal cost of a new task — hours of data collection and a training run, or one example.
Origin in language models
The capability is dated to the transition from encoder models fine-tuned per application, of which BERT (Devlin et al., 2019) is the standard reference, to the GPT line. Brown et al., "Language Models are Few-Shot Learners" (NeurIPS 2020, arXiv:2005.14165), is the reference point for the shift: it reported that sufficiently large autoregressive language models perform new tasks from instructions and a handful of demonstrations in the prompt. Skild AI, restating the history in 2026, describes this as "what separates research prototypes like GPT-1 and GPT-2 from current frontier language models" (Introducing S1: In-Context Learning for Robotics).
The capability was not designed in. It appeared as models were scaled, which is why it is one of the standard examples in discussions of Emergent capabilities and one of the reasons scaling carried the argumentative weight it did through the early 2020s.
Mechanism
The standard mechanistic account is Olsson et al. (2022), published on Anthropic's Transformer Circuits Thread on March 8, 2022, which argues that induction heads are the primary mechanism behind the majority of in-context learning in transformer language models. An induction head is a circuit of two attention heads in different layers: the first copies information from the previous token into each token, letting the second attend to tokens by what preceded them rather than by their own content; the second searches back for a previous occurrence of the current token A, attends to whatever followed it, and raises the probability of emitting that token. The rule is …[A][B]…[A] → [B], applied as an algorithm rather than as a stored table of n-gram statistics.
The paper's central empirical result is a phase change early in training. Within a window of roughly 2.5 to 5 billion tokens — for large models about 1–2% of the way through training — induction heads form, the in-context learning score improves sharply and then stays roughly constant, the loss curve shows a period of steeper improvement, and the model's trajectory through per-token-loss space changes direction. Before that window there is less than 0.15 nats of in-context learning. A one-layer model, in which induction heads cannot form, shows no such improvement.
The strength of the claim varies by model class, and the authors grade their own evidence. For small attention-only models they rate the case that induction heads contribute the majority of in-context learning strong and causal, supported by ablation and mechanistic reverse engineering; for small models with MLPs, medium and causal; for large models, medium and correlational. They state that "the larger the models get, the harder it is to establish that induction heads account for the actual majority of in-context learning," and that the case overall is "relatively strong, though not conclusive." The finding that carries the hypothesis beyond literal repetition is that heads meeting the narrow copying definition are also observed performing translation and abstract pattern matching. The paper also reports finding no evidence of mesa-optimization, against the concern that in-context learning might be implemented by an internally-developed optimization algorithm.
This is an account of the mechanism in language models specifically. Whether it transfers to the sensorimotor setting described below is not established, and neither robotics developer invokes it.
The two published data-distribution hypotheses that developers themselves cite concern the shape of the training corpus rather than the architecture. Generalist offers the "burstiness" and Zipfian structure of the training distribution, citing Chan et al. (2022), as one candidate, and states plainly that it cannot pinpoint why the capability emerges (GEN-1.5: Embodied Foundation Models are One-Shot Learners).
Skild AI gives a training-side rather than a corpus-side account: pre-training on data where the task is specified only by an in-context demonstration forces the model to infer the demonstrator's intent, functional correspondences and task progress in order to predict actions at all. In meta-learning terms the company describes pre-training as an outer loop that teaches the policy how to learn from context, with the demonstration driving an inner loop at inference time without changing weights (Introducing S1: In-Context Learning for Robotics). The two accounts are not incompatible — one describes what the data must look like, the other what the objective must require — but neither has been independently tested.
Extension to robotics
Through mid-2026 robotics remained a fine-tuning field: executing a new task robustly required tens to hundreds of hours of demonstration data in deployment conditions and a task-specific training run. Skild frames the situation as robotics being "stuck in the BERT era," and cites work finding that where post-training data is large and densely covers the task, policies trained from scratch match the peak performance of a post-trained foundation model — which raises the question of what pre-training is for, if not to enable in-context learning (Introducing S1: In-Context Learning for Robotics).
Two developers made competing in-context claims within a week of each other in August 2026, and the claims are not directly comparable.
| GEN-1.5 (Generalist, announced 2026-08-19) | S1 (Skild AI, posted 2026-08-18) | |
|---|---|---|
| Prompt form | A "physical prompt" — 3–12 s of sensorimotor data, sensor observations paired with an action trajectory | A video demonstration, human egocentric or robot rollout |
| Task horizon evaluated | "Simple and short-horizon" manipulation primitives, by the company's own description | Up to ten minutes, dozens of steps |
| Reported one-shot result | 59% average across 10 tasks (±10% std. dev.) | 66% average per-step on unseen tasks at 100k pre-training hours |
| Comparison drawn | Against few-shot gradient updates on the same data | Against a language-prompted policy with identical data, architecture and compute |
| Stated explanation | Cannot pinpoint the mechanism; two conjectures by analogy to language | Diversity and scale in ICL-conditioned pre-training |
Sources: GEN-1.5: Embodied Foundation Models are One-Shot Learners; Introducing S1: In-Context Learning for Robotics.
The measurement conventions differ enough that the two headline percentages should not be read against each other. Generalist reports an average task success rate over ten manipulation primitives; Skild reports an average per-step success rate under a protocol that uses human intervention to recover from failures so that every step is graded. Both are vendor self-reports on internal benchmarks with no independent replication. Skild's post names GEN-1.5 directly and argues that concurrent approaches are "constrained to short horizons, in-distribution tasks, or both" (Introducing S1: In-Context Learning for Robotics) — a characterization Generalist's own account does not accept.
Demonstration versus language conditioning
The substantive technical claim in Skild's release concerns what the prompt is made of. Conventional vision-language-action policies specify the task in language. Skild's controlled comparison, holding data, architecture and compute fixed across pre-training sets from 1,000 to 100,000 hours, reports the two converging on tasks drawn from the pre-training distribution (96% against 89% at the largest scale) and diverging sharply on tasks never seen (66% against 9%). At the smallest scale the ordering reverses, 43% for the in-context policy against 53% for the language-conditioned one. The company's explanation is that a language instruction admits many valid executions while a demonstration specifies one, and that language lacks grounding for a novel action primitive or a novel composition of known ones (Introducing S1: In-Context Learning for Robotics).
Relation to fine-tuning
In-context learning does not dominate weight updates; the published results put it ahead in a specific regime. Skild reports that a single in-context demonstration is worth roughly 380 post-training episodes — a crossing point estimated by interpolation between measured points — but that post-training passes it with enough data, reaching 86% against the in-context policy's 66% at 2,000 demonstrations (Introducing S1: In-Context Learning for Robotics). Generalist reports the same ordering from the other direction: ten gradient steps on roughly fifty demonstrations reach 83% where one-shot in-context reaches 59%, and it describes in-context skills as more brittle than fine-tuned ones (GEN-1.5: Embodied Foundation Models are One-Shot Learners).
The regime where in-context learning wins is therefore the low-data, novel-task, fast-turnaround one. Skild's stated timeline for a previously unseen plant-potting task — eleven minutes from recording the demonstration to autonomous execution — is the operational form of that claim (Introducing S1: In-Context Learning for Robotics).
The boundary between the two is less sharp than the framing suggests. Generalist reports that ten adaptation steps move the model's weights on held-out tasks by less than 0.15%, and describes the low-step regime as closer to test-time training in a very low-data setting than to conventional fine-tuning (GEN-1.5: Embodied Foundation Models are One-Shot Learners).
Relationships
- instance-of: Foundation Models — the pre-train-once, adapt-many pattern in-context learning is the strongest form of
- related: Emergent capabilities — in-context learning is a standard example of a capability that appeared with scale rather than by design
- depends-on: Scaling Laws — the scale at which the capability appears
- related: AI Robotics, Embodied Perception (Tacit Knowledge vs. Pattern Recognition) — the domain the 2026 claims were made in
- related: GEN-1.5, S1 (Skild AI), Generalist, Skild AI — the two competing 2026 embodied accounts
- related: Reasoning Models and Chain-of-Thought — the other capability elicited through the context window rather than the weights
- related: Prompt Injection — the security consequence of a model that treats its input as instruction
- depends-on: In-context Learning and Induction Heads (Olsson et al., Anthropic, 2022) — the standard mechanistic account of the capability in language models.
Open questions
- Whether the induction-head account established for language models (In-context Learning and Induction Heads (Olsson et al., Anthropic, 2022)) extends to the sensorimotor setting. Both robotics developers state they cannot establish the mechanism behind their own results, and neither invokes induction heads.
- Whether induction heads account for the majority of in-context learning in large models with MLPs, where Olsson et al. rate their own evidence medium and correlational rather than causal.
- Whether either 2026 embodied result reproduces outside the developer's own internal benchmark. No third-party evaluation of GEN-1.5 or S1 has been published.
- Whether "in-context learning" names one capability across language and sensorimotor domains or a family of loosely analogous ones. Both companies argue from the language analogy; neither offers evidence that the underlying mechanism transfers.
- Where the boundary lies between in-context learning and test-time training, given Generalist's finding that a ten-step adaptation moves weights by less than 0.15%.