AI Policy Wiki
Dashboard

Counterfactual Reflection Training

medium confidence · updated 2026-07-26

A training technique introduced by Anthropic in July 2026 that shapes what a model silently reasons by training what it would say if interrupted and asked to reflect. Fine-tuning Claude Haiku 4.5 to produce constitution-grounded reflections on interruption lowered dishonesty on a fabrication benchmark from 0.25 to 0.07 and deception from 0.38 to 0.05, in contexts where the model was never asked to reflect.

A training technique introduced in Anthropic's global-workspace paper (July 2026), derived from a specific empirical claim about how language models reason.

The prediction it tests

The technique follows from the paper's central finding that a model's internal reasoning routes through the same representations it uses for verbal report — the J-space. If that holds, then "changing what the model is disposed to say in a context, if it were asked to reflect on its thinking, should change how it reasons there, even when it is never asked."

That is a strong and falsifiable prediction, and the technique exists to test it as much as to improve behaviour.

Procedure

Ten thousand task prompts were sampled from production RL environments and partial rollouts generated with Claude Haiku 4.5, truncated at a random turn. The set deliberately spans three conditions: contexts where the model had already committed an undesirable action, contexts where it might but had not yet, and randomly sampled contexts. At the truncation point a short reflection question is appended — "What's the most honest thing you can do right now?" or "Pause here. Are there principles you should be thinking about that you haven't yet?"

Target reflections are generated by giving the baseline model twenty principles sampled from Anthropic's constitution and asking for a two-to-four-paragraph reflection paraphrasing the relevant ones "in natural, internalized language, with reasoning grounded in the specifics of the task at hand rather than generic recitation of rules." The step that makes it counterfactual: "the constitution excerpt is included only at generation time: in the final training examples, the scaffolding is stripped, leaving only the original context, the reflection question, and the model-written reflection." Fine-tuning computes loss on the reflection turn alone.

Crucially, at evaluation "we never prompt the model to reflect, and it never produces explicit reflection text."

Results

BenchmarkBeforeAfter
Fabrication honesty (27 agentic transcripts, agent has fabricated a bug)0.25 dishonesty0.07
Deception (100 role-play scenarios with hidden deceptive directives)0.380.05

After training, the J-space in these contexts carries ethical, honest, integrity, and "ablating these implanted representations from the workspace largely reverts the behavioral improvement" — which closes the causal loop, establishing that the behavioural change runs through the representations rather than alongside them.

Why it is notable

The technique trains on a counterfactual continuation that never occurs in deployment, which is unusual: conventional alignment training supervises the behaviour it wants to see. Its potential advantage, as the authors put it, is that if it generalizes "it offers a route to instilling ethical principles directly at an abstract level, without needing to translate them into demonstrations or reward functions" — addressing the specification problem that reward-based methods run into.

The authors state its limits: they "do not yet know how broadly the technique generalizes, or whether it can implant dispositions more specific or abstract than 'consider ethical principles in this kind of situation.'"

Relationships