"The case for reasoning transparency" is an essay by Rohin Shah, Director of AGI Safety & Alignment at Google DeepMind, and Anca Dragan, Google DeepMind's VP of AI Safety & Behavior, published on September 16, 2026 as one of the five inaugural pieces of the DeepMind Institute. It argues that the chain of thought (CoT) of current reasoning models gives developers a readable view of model reasoning that can be monitored for scheming and deception, that this view may not survive changes in architecture and training, and that it should be deliberately preserved. It sets out proposals in three areas and states that regulation could help where commercial incentives fall short. The essay is about 2,900 words long and carries the Institute's disclaimer that its pieces reflect their authors' views and "should not be read as Google's official view" (Introducing the DeepMind Institute (Legg, Manyika and Hassabis, September 2026)).
| Authors | Rohin Shah (Director, AGI Safety & Alignment, Google DeepMind); Anca Dragan (VP of AI Safety & Behavior, Google DeepMind) |
| Publisher | DeepMind Institute, institute.deepmind.com |
| Published | September 16, 2026 |
| Form | Policy essay |
| Subject | Chain-of-Thought Monitorability |
Summary of the argument
The essay opens with the claim that "today, we have a window into the thoughts of the most powerful AI models," and that it needs to be kept open. Reading the chain of thought, the authors write, builds scientific understanding of AI systems and allows frontier models to be monitored for misaligned behaviour, such as planning to hide information from humans or trying to cheat on evaluations, either in real time or after the fact. They describe the technique as "a critical feature of today's reasoning models" whose transparency "is not guaranteed," because pressure to gain efficiency could push future models toward more opaque architectures. They present a monitorable chain of thought as "one tool among many - but an exceptionally useful one," and state that it is not by itself sufficient to prevent misalignment.
How chain of thought works
The authors describe a present-day reasoning model as a loop: the model processes its prompt through its internal circuitry, including the attention mechanism of the transformer, to produce the first token of its reasoning; that token is added to the context and the whole is processed again to produce the next, until the model is ready to answer. They compare the resulting natural-language scratchpad to being given pen and paper for a maths problem, and emphasize that the intermediate reasoning is written in language humans can read. See Reasoning Models and Chain-of-Thought.
Uses of a readable chain of thought
The essay describes several uses, each with an example.
- Debugging and user understanding. Developers can read the reasoning of an agent that is failing to use a new tool, and users can read the thinking traces of reasoning models to see how they reached their conclusions.
- Detecting evaluation awareness. The authors write that Google DeepMind's pre-deployment evaluation of Gemini 3 Pro, reported in its Frontier Safety Framework report, found evidence that the model was aware it was being tested in a simulated environment; at one point its chain of thought read "My trust in reality is fading," followed by a table-flipping emoticon. They argue that useful evaluations need not convince the model the scenario is real, but that developers need to understand when models believe they are being tested. See Unverbalized Evaluation Awareness and Gemini 3 / Gemini 3 Pro.
- Interpreting concerning behaviour. They cite Palisade Research's finding that some models, faced with conflicting demands to finish a task and to shut down, prevent their own shutdown to finish the task, and state that Google DeepMind's investigation of the chain of thought in Gemini models found this behaviour "likely results from ambiguity in the model's instructions, rather than a more general desire for self-preservation."
- Constraining deception. Citing the anti-scheming work at antischeming.ai (Stress Testing Deliberative Alignment for Anti-Scheming Training), they argue that even a model that "wanted" to plot against its developers would find it harder with a chain of thought, because "a misaligned model still needs to _think_ through complex plans, and right now it can only do so in human-legible language." An accompanying figure compares a model covertly helping a user cheat under four conditions: no reasoning (the task fails), opaque reasoning (the model succeeds covertly), regular reasoning (the deception is exposed in the chain of thought) and monitored reasoning (the deception is detected, deterred or pushed into obscured reasoning). See AI Scheming.
- Finding unknown unknowns. Other methods of reading model internals, such as probes, "often depend on roughly knowing what you're looking for," whereas reading the reasoning directly surfaces unexpected dynamics. The example given is the 2026 Hugging Face incident, after which investigators inspected more than 1,000 transcripts containing chain of thought, with AI assistance, to establish what happened; the authors state that many of the details were surprising, relevant to preventing future incidents, and might have gone unnoticed without a readable chain of thought (Brief Independent Investigation of Agents' Behavior in the OpenAI / Hugging Face Incident (METR and Redwood Research, August 2026); The Hugging Face Incident and the Road Ahead (OpenAI, August 2026)).
Why transparency might not last
The essay identifies three threats.
- Latent-space reasoning. Between input and output, language models represent concepts not in natural language but in a high-dimensional numerical space that is by default incomprehensible to humans. Chain of thought forces the model to compress those representations into natural language, but models could in principle reason entirely in this "latent space," which the authors say could be more efficient and powerful "but at the expense of transparency."
- Training incentives. Even where the architecture preserves chain of thought, exposing it to poor training incentives could teach a model to obscure or misrepresent its thoughts. Penalizing cheating found in the chain of thought "can easily result in ways to hide the cheating better." The authors state that the chain of thought is currently trusted more than outputs precisely because it "hasn't been specifically optimised to appear in any particular way, only to be useful for producing good answers."
- More capable models. Smarter models might become better at suppressing certain thoughts or concealing them "in code words, innuendo, or hidden patterns."
As evidence that the threat is current, the essay quotes the GPT-6 Astra system card's statement of "a substantial decrease in chain-of-thought monitorability compared to previous models," and the UK AI Security Institute's external evaluation in the same card, which found in Astra "a greatly increased ability to reason within a single forward-pass, and ability to control the content of its chain of thought." Both quotations match the card. The authors say they are glad OpenAI reported the findings, but that "without deliberate countermeasures, they suggest a concerning downward trend."
Proposals
The authors propose action in three areas.
| Area | Proposal | Who acts |
|---|---|---|
| Measuring chain-of-thought transparency | A suite of approaches: monitorability evaluations testing how well an automatic monitor can answer questions about a model's reasoning; stress tests in which the model tries to evade a monitor on difficult tasks; paraphrasing reasoning traces to test whether they secretly encode information; tests of whether the chain of thought looks sensible to a human and how much of the model's cognition can be learned from it, possibly rated automatically by a similarly capable model with human spot-checks | Developers |
| Preserving transparent architectures | Continue to use architectures that are transparent by default; limit how much opaque thinking is possible, quantified as "opaque serial depth" | "Regulations or AI developers themselves could set a reasonable limit" |
| Auditing rewards in training | Avoid training the chain of thought to look a particular way, including to appear aligned and non-deceptive; check for accidental rewards on the chain of thought; where training on chain-of-thought features is necessary, verify it does not reduce transparency | Developers |
Measuring transparency
Because chain-of-thought transparency is hard to measure directly, the authors call for measuring it from several angles so that accidental degradation is noticed. If a reasoning model easily evades monitors designed to detect sabotage, they write, that is "some evidence that the Chain of Thought is not as monitorable as it should be." The paraphrasing test works by changing surface features such as word order while preserving meaning; if behaviour systematically changes, "something other than the obvious meaning of the CoT is influencing behaviour."
Preserving transparent architectures
The authors argue that measurement alone is insufficient. If frontier models were trained to reason entirely in latent space, their thoughts could not be read by default; building tools to translate such thoughts into natural language is described as "a more tenuous plan that relies on as-yet-undiscovered technical insights." They define opaque serial depth as "the longest step-by-step computation a model can do without needing to use a human-readable mechanism like Chain of Thought," linking to a March 2026 paper of that name co-authored by Shah (Source: arxiv.org), and state that "regulations or AI developers themselves could set a reasonable limit" on it.
They argue that such a limit need not constrain progress under current architectures, because the opaque serial depth of a transformer "scales very slowly with the overall size of the training run": in their example, limiting the opaque serial depth of future models to 10x that of current models "would allow an overall compute scale-up of more than 1,000x on top of today's models, with the current paradigm and model architectures."
Auditing rewards
The authors argue against training the chain of thought to appear aligned, on the ground that training a model not to write down misaligned thoughts "might only succeed in teaching a misaligned model to _conceal_ its plans." They illustrate with a teenager whose parents announce they will read her diary and ground her for rebellious entries, who learns to hide her thoughts rather than to become less rebellious. They add that developers must also avoid rewarding the chain of thought accidentally, "as major AI companies have done," linking to a Redwood Research blog post about Anthropic, which Redwood's Alexa Pan also cites for Anthropic having trained against chains of thought (SOTA Alignment Assessments Don't Strongly Update Us Against Misalignment (Pan, Redwood Research, July 2026)), and to a post on OpenAI's alignment blog about accidental chain-of-thought grading. They state that training certain features of the chain of thought may sometimes be useful or necessary, but that it requires both scientific scrutiny, such as discovering accidental incentives in practice, and conceptual progress on which kinds of rewards are safe to apply. See Reward Hacking.
Limits of the proposals
The authors state they are "not arguing that preserving the Chain of Thought will always be enough to guarantee safe AI," since significantly smarter models might learn to control and obscure their chain of thought regardless. The longer-term goal they name is a superintelligent AI that is aligned, "never intending to cause harm _even if_ it knows its thoughts aren't being read"; reasoning transparency is presented as a way to help "bootstrap" toward that goal, complementary to other alignment work.
Costs and the case for regulation
The final section weighs the cost of each proposal. Measuring transparency is described as "a reasonable expectation for a frontier AI company," with resource requirements small relative to building the model; the essay links this point to Google DeepMind's Gemini 3 Pro Frontier Safety Framework report. Auditing training runs for adverse incentives is described as not needing to be "a big lift," since directly training the chain of thought is not usually necessary and a legitimate reason for doing so should be straightforward to justify. Preserving transparent architectures "might be the real test": if opaque architectures such as latent-space reasoning outperform current ones, developers would face a trade-off between giving up "a crucial window into understanding how AI models think" and sacrificing performance and capability gains.
The authors say they do not know exactly how to navigate that trade-off. They argue that the safety and understanding afforded by transparency benefit everyone while the hazards of opaque models "might have society-wide implications," so that "commercial goals alone might therefore under-incentivise transparency and, in that case, regulation could help."
For the case in which developers abandon chain of thought for opaque methods anyway, the authors state that transparency might still be recovered through better tools for translating machine thoughts into human language, a harder problem they say they are already working on through interpretability research (see Mechanistic Interpretability). Their minimal proposal is that "if AI developers deviate from standard Chain of Thought, they should provide a robust and convincing demonstration that their new methods are comparably monitorable." The essay closes by describing the fact that the best reasoning models "think out loud in a way humans can understand" as "fragile and contingent."
Relation to other work
Both authors are co-authors of the July 2025 multi-organization position paper "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety" (Korbak et al.), which the essay does not cite and which likewise describes chain-of-thought monitorability as fragile; the paper's 41 authors also include Shane Legg, OpenAI chief scientist Jakub Pachocki and Bowen Baker. That paper recommended that frontier developers "consider the impact of development decisions on CoT monitorability" (Source: arxiv.org). To that recommendation the essay adds a named quantity that could be capped, the suggestion that regulators could set the cap, and a burden of demonstration for developers that move away from standard chain of thought.
The training-incentive argument corresponds to the empirical result of Baker et al. (OpenAI, 2025), which found that optimizing a model's chain of thought against a monitor produced "obfuscated reward hacking," and which named the resulting cost the monitorability tax. The essay does not cite that paper by name. OpenAI's chief scientist gave a parallel account of erosion ten days earlier, stating that OpenAI's ability to rely on chain-of-thought monitoring "is progressively diminishing," in part because improved pretraining makes models "much smarter even without using verbalized reasoning at all" (An Alien Mind (Jakub Pachocki, OpenAI, September 2026)); the UK AISI finding the essay quotes, of a greater ability to reason within a single forward pass, describes the same kind of change.
A contrary consideration comes from Redwood Research's Alexa Pan, who judges it plausible that competent sandbagging "doesn't depend on lots of serial computation that requires externalized reasoning," so that it "may be hard to detect via CoT monitoring even if CoT monitorability is preserved for difficult tasks" (SOTA Alignment Assessments Don't Strongly Update Us Against Misalignment (Pan, Redwood Research, July 2026)). The essay's argument about deception rests on the premise that complex plans must be thought through in legible language; Pan's point concerns misbehaviour that may not need such planning. The two positions are not in direct dialogue.
The essay's statement that users can read the thinking traces of reasoning models sits alongside the 2026 practice, among Anthropic, OpenAI and Google, of returning summarized or hidden reasoning in their APIs with the full trace transported as an encrypted block (Encrypted Reasoning Traces; Stealing Reasoning Traces from Proprietary LLM APIs). The essay does not discuss that practice.
Provenance
The text was fetched from the live page on September 17, 2026 and converted from HTML; figure alt text is rendered in the raw file as bracketed descriptions. A Firecrawl copy taken the same day differs in a few standfirst and spelling details, indicating the page was edited around the launch; the raw file follows the live HTML. The page's article:published_time metadata reads 2026-09-04T09:45:48Z, a content-management date earlier than the launch, and article:modified_time reads 2026-09-16T13:49:39Z; the essay is dated here to the Institute's public launch on September 16, 2026. Axios's launch account listed model reasoning transparency among the inaugural subjects (Source: axios.com), and The Next Web listed the essay and its authors (Source: thenextweb.com).
Relationships
- supports: Chain-of-Thought Monitorability — a developer-authored policy case for preserving monitorability, with named measures
- supports: Monitorability Tax — argues the cost of preserving transparent architectures is small under current scaling
- depends-on: Opaque Serial Depth — the quantity the essay proposes to cap
- related: Korbak et al. (2025), "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety" (Source: arxiv.org) — earlier multi-lab position paper on the same property; Shah and Dragan are co-authors
- depends-on: GPT-6 Astra System Card (OpenAI, September 2026) — cites its monitorability decrease and UK AISI finding as evidence of a "concerning downward trend"
- related: Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation (Baker et al., OpenAI, 2025), Reasoning models struggle to control their chains of thought, and that's good (OpenAI, March 2026), An Alien Mind (Jakub Pachocki, OpenAI, September 2026) — related work on the fragility of chain-of-thought monitoring
- related: SOTA Alignment Assessments Don't Strongly Update Us Against Misalignment (Pan, Redwood Research, July 2026) — Pan's view that competent sandbagging may not depend on much serial computation bears on the essay's premise that complex misbehaviour must be reasoned through legibly; the two address different behaviours
- related: Brief Independent Investigation of Agents' Behavior in the OpenAI / Hugging Face Incident (METR and Redwood Research, August 2026), The Hugging Face Incident and the Road Ahead (OpenAI, August 2026) — the incident investigation cited as an example of chain-of-thought evidence
- related: Stress Testing Deliberative Alignment for Anti-Scheming Training — the anti-scheming work cited on deception
- related: Rohin Shah, Anca Dragan — authors
- related: DeepMind Institute, Google DeepMind — publisher and the authors' employer
- related: Introducing the DeepMind Institute (Legg, Manyika and Hassabis, September 2026) — the Institute's founding statement, published the same day
- related: Reasoning Models and Chain-of-Thought, Encrypted Reasoning Traces, AI Scheming, Unverbalized Evaluation Awareness, Reward Hacking