A safety case is a structured, documented argument — supported by evidence — that a system is acceptably safe to operate for a defined purpose in a defined environment. The concept is long-established in high-reliability industries: civil aviation, nuclear power, rail, offshore oil, and defense all require a safety case before a system is certified. Its migration into frontier AI governance reframes the question from "did the developer follow a checklist?" to "can the developer affirmatively demonstrate this model will not cause unacceptable harm?"
Contrast with procedural governance
Most early AI governance is procedural — it specifies steps a developer must take (run these evaluations, publish a model card, adopt a policy). A safety case is outcome-oriented: the burden is on the developer to construct a positive argument that the residual risk is acceptable, and to expose that argument to scrutiny. This shifts the default: absent a credible safety case, deployment is not justified.
Structure of an AI safety case
A frontier-AI safety case typically combines several argument types:
- Inability — the model lacks the dangerous capability altogether (supported by capability evaluations and red-teaming).
- Control — even if capable, the model cannot exercise the capability because deployment safeguards, monitoring, or affordance limits prevent it.
- Trustworthiness — the model is capable and unconstrained but will reliably not misuse the capability (the hardest case to make, and the one most dependent on interpretability and alignment evidence).
- Deference — reliance on the judgment of a credible external body.
Each argument is only as strong as its evidence base, and the genre is explicit that current evidence — especially for trustworthiness — is weak.
Templates and research program
The concept has moved from proposal to worked artifacts. The paper "Safety Cases: A Scalable Approach to Frontier AI Safety" (Buhl et al., 2025) sets out how and why frontier developers might adopt the practice (Source: researchgate.net). The UK AI Security Institute has published a safety-case template for inability arguments, defining safety cases as "clear, assessable arguments that show that a system is safe in a given context" (Source: aisi.gov.uk), and the Centre for the Governance of AI published a template for a cyber inability argument structured in the Claims-Arguments-Evidence (CAE) framework to make safety arguments coherent and explicit (Source: governance.ai). For the harder control case, a published sketch argues models can be deployed safely because of measures such as monitoring and human auditing of suspicious behavior (Source: lesswrong.com). A March 2026 arXiv paper, "Rethinking the Foundations of Frontier AI Safety Cases," surveys this program — including the UK AISI's cyber-inability and control agendas — and argues for revisiting its foundational assumptions (Source: arxiv.org).
Relationship to other frameworks
Safety cases are increasingly the mechanism inside the if-then commitments of a Responsible Scaling Policy (RSP) and its peer AI Safety Cases and Frameworks: at higher capability tiers, the required safeguard is the production of a reviewed safety case. UK government work and bodies such as UK AI Safety Institute (AI Security Institute) and Apollo Research have promoted safety cases as a near-term, auditable governance artifact.
Developer definitions and independent assessment
OpenAI published its own definitions in September 2026, in a post proposing that independent third parties assess its safety cases (Priorities and Principles for Effective Third Party Assessments (OpenAI, September 2026)). It defines a safety case as "a structured argument, supported by evidence, explaining why a model or system's risks are adequately managed for a specified activity, such as training, evaluation, or deployment," which "connects individual safety claims to the evidence supporting them and makes explicit the assumptions, uncertainties, and remaining risks," and a safety claim as "a specific assertion about a model or system's capabilities, behavior, or safeguards that bears on its safety and can be assessed against evidence." The definition differs from the one used by Buhl et al., taken from the UK Ministry of Defence — "a structured argument, supported by evidence, that a system is safe enough in a given operational context" (Safety Cases for Frontier AI) — in using "adequately managed" risks as the standard and in scoping a case to an activity, including training and internal deployment, rather than to an operational context.
The first of the post's four priority areas is "independent assessment of safety cases, spanning training, evaluation, internal deployment and external deployment," by multiple assessors examining different parts of a case. Its questions include whether a case's evidence is substantiated, whether its conditions were followed, whether it covers "the most urgent risks identified in the course of an assessment," and whether training methods reduce incentives that could reward deception or reward hacking. Assessment begins from safety claims pre-registered and agreed between lab and assessor. See Third-Party AI Assessment.
Relationships
- instance-of: AI Safety Cases and Frameworks
- depended-on-by: Responsible Scaling Policy (RSP) — higher ASLs require an affirmative safety case
- related: AI Existential Risk, Mechanistic Interpretability, Apollo Research, UK AI Safety Institute (AI Security Institute)
- supported-by: Safety Cases for Frontier AI
- related: Priorities and Principles for Effective Third Party Assessments (OpenAI, September 2026) — OpenAI's September 2026 definitions of safety claim and safety case, and its proposal for independent assessment of its safety cases
- related: Third-Party AI Assessment