By mid-2026 the question of whether frontier AI developers should face outside review had largely been settled in the affirmative across US federal proposals, state statutes and the developers' own published positions. The unsettled question is institutional: who selects the reviewer, who pays them, who accredits them, and what happens when they find something. Five distinct answers are in play, and they are usually argued past one another because each is developed within a separate literature — statutory drafting, private-governance theory, industry self-regulation, national-security evaluation, and tort scholarship. A sixth design, published in August 2026, deliberately crosses two of them and is treated separately below.
This page sets the five side by side, then treats the crossing design and the position that rejects pre-release review altogether. It does not compare the content of safety frameworks, which is treated at AI Safety Frameworks Compared: RSP, Preparedness, NIST RMF, and Safety Cases, nor the government-review question in isolation, which is treated at AI Pre-Release Vetting.
The five architectures
| Statutory mandatory audit | Licensed private verifiers (IVOs) | Industry self-regulatory body | Direct government evaluation | Mandatory liability insurance | |
|---|---|---|---|---|---|
| Reference instrument | Illinois SB 315 | FRONTIER Act; Connecticut SB 5 pilot | Hassabis FINRA-style body (A Framework for Frontier AI and the Dawning of a New Age (Hassabis, July 2026)); Google's FARO proposal | CAISI pre-release evaluation; the June 2 executive order's framework | Weil's proposal (Don't Let AI Developers Hire Their Own Referees (Weil, July 2026)) |
| Who selects the verifier | The developer, from qualified auditors | The developer, from licensed IVOs | The body assigns, from its own staff | The government | The developer, from insurers |
| Who pays | The developer | The developer | Industry, by levy | The taxpayer | The developer, as premium |
| Who accredits | Not specified in the enacted text | Commerce Department (FRONTIER Act); state consumer-protection department (Connecticut) | The body's majority-independent board | Not applicable | Standard insurance solvency and conduct regulation |
| Trigger | Annual, for covered frontier developers | Periodic adequacy audits | Voluntary submission up to 30 days pre-release, mandatory later | Voluntary submission for models meeting a classified capability threshold | Continuous, for the life of the policy |
| Access | Not specified | Not specified in any of the four instruments | Not specified | Classified benchmarking; onward sharing with federal agencies and trusted corporate partners | Whatever the insurer negotiates |
| Public disclosure | Public safety-practice disclosure; incident reporting | Auditor-facing rather than public | Not specified | The framework itself is unpublished; benchmarks and thresholds classified | Weil proposes disclosure of insurer risk assessments with liability for false ones |
| Consequence of failure | Civil penalties up to $3 million | Civil penalties; in some versions a liability shield for passing | Later versions: no US-market deployment | No formal bar on release; leverage runs through export-control threat | Claims paid; renewal conditioned on fixes |
The rows left blank are the finding rather than an omission. No instrument in any lane specifies what a verifier may run, read, or ask. The most concrete published answer comes from METR, outside all five lanes, in its proposal for post-incident investigation: the ability to run all models involved, full transcripts or reproducible environments, employee interviews across three named staff groups, and prompted classifiers over the training data, with training-data ablations and intermediate-checkpoint access needed to establish whether a behavior traces to reinforcement-learning trajectories (How independent researchers could investigate AI propensities after misalignment incidents (METR, July 2026)).
The selection problem is the axis
Four of the five lanes leave the reviewed party choosing its reviewer. That shared feature, rather than the differences in accreditation or penalty, is what the substantive disagreement turns on.
OpenAI named the problem and accepted the structure in the same sentence on August 7, 2026: "Companies should be able to choose among qualified assessors, but the system should guard against 'assessor shopping' designed to avoid or soften adverse findings" (Making AI Audits and Assessments Work (OpenAI Global Affairs, August 2026)). Its proposed guards are accreditation on documented criteria, disclosure and management of material conflicts, a bar on assessing one's own work, and a bar on outcome-contingent compensation.
Gabriel Weil argues the guards are insufficient because the incentive runs the other way: an IVO dependent on the developers it clears for repeat business has an incentive to grade gently, so "competition, the feature that is meant to make the IVO model effective, instead drives it toward laxity" — the credit-rating-agency failure of 2008 (Don't Let AI Developers Hire Their Own Referees (Weil, July 2026)). His answer changes who holds the pen rather than adding conditions to the selection: an insurer's capital is exposed for the life of the policy, so it has a principal that loses money when the assessment is wrong. He grants the design's limit — for extreme low-probability catastrophes the insurer is itself judgment-proof, so those risks never reach the premium — and confines the requirement to the "insurable layer." A fuller treatment of the developer-pays criticism is at Independent Verification Organizations (IVOs).
The industry-body lane escapes the selection problem by assigning the reviewer, and inherits an enforcement problem instead. Zvi Mowshowitz's objection to the FINRA analogy is that "you need an SEC to your FINRA," and that a voluntary regime exempting internal deployment is inadequate; Andy Hall's is that a scheme built on voluntary lab submission has no purchase on open-weight releases (AI Pre-Release Vetting).
The government lane escapes it entirely and has the weakest published standards. As of August 2026 the framework required by the June 2 executive order was complete but unpublished, its benchmarking process and covered-model threshold classified, with no person or organization designated to lead outreach (AI Pre-Release Vetting). The objections to it are procedural rather than substantive: Brad Carson's "if only tech companies know what's in the rulebook, it doesn't work," and five Senate Democrats' complaint that "even justifiable interventions can create broader harm if the standards and decision-making processes are opaque, ad hoc, or unpredictable" (Senate letter on the Administration's approach to limiting access to advanced AI models (Gillibrand, Warner, Kelly, Schiff, Coons, August 2026)).
What the lanes agree on
Three points recur across instruments that otherwise share no design premise.
A control framework must precede the audit. OpenAI states it directly — "Requiring companies to undergo review is not enough. A defined control framework is a prerequisite for effective audits" — and asks for common standards defining risks and controls in scope, the evidence and criteria reviewers use, the assurance level expected, and reporting form (Making AI Audits and Assessments Work (OpenAI Global Affairs, August 2026)). The same premise underlies the NIST AI Risk Management Framework's role in US instruments and the drafting of NIST AI 200-2, which supplies a method for constructing an assessment from an organisation's own objectives rather than a fixed benchmark suite.
Public reporting is summary, not disclosure. Where a lane specifies public output at all, it converges on a plain-language summary with targeted redactions rather than release of underlying evidence. OpenAI's list — scope, applicable standards, methodology, assurance level, material findings and limitations, required corrective actions, remediation status, plus the auditor's identity, qualifications and material conflicts — is the most detailed published version, paired with complete reports to regulators through protected channels (Making AI Audits and Assessments Work (OpenAI Global Affairs, August 2026)). The FRONTIER Act's disclosure regime is likewise auditor-facing rather than public.
Interoperability is the stated reason for federal primacy. OpenAI's argument that "comparable risks should be evaluated using rigorous, interoperable criteria and methods" runs parallel to its preemption ask (Democratic Governance of Frontier AI: A blueprint for a federal framework (OpenAI, June 2026)), and the two are hard to separate: an interoperability requirement satisfied by a national baseline is also an argument against divergent state audit regimes. Illinois SB 315, the one enacted mandatory-audit statute, would be displaced by the federal duty the same company proposes (Illinois SB 315 (frontier safety framework with mandatory third-party audits)).
A sixth design crosses two lanes
The ARI blueprint of August 10, 2026 does not sit in any single lane, and its design is an explicit answer to the selection problem above (Responsible Innovation at the Frontier (ARI, August 2026)). It begins in the direct-government-evaluation lane — the government conducts all assurance examinations, funded and staffed for the purpose, with federal evaluation teams testing systems against the standards in force and a standing supervision team modeled on the Federal Reserve's oversight of the largest banks examining safety practices. It moves toward the licensed-private-verifier lane only through a three-phase shift, and only in risk domains where the regulator certifies against fixed criteria that accredited capacity exists.
Three features distinguish it from the four developer-selects lanes:
- The delegable question is narrowed. Only compliance assurance — a technical determination against a fixed published standard — may be delegated. Adequacy assurance, the judgment whether a developer's framework is sufficient, is never delegated at any level of market maturity, on the reasoning that it feeds directly into the next standards cycle.
- Selection is split, then shared. In Phase Two the developer selects one accredited verifier per domain and the government examines alongside it; in Phase Three the developer selects one and the government assigns the other. In every domain, cycle and phase the government retains a share of examinations as a capability floor, which the authors justify as preserving the regulator's ability to reproduce verifier work and to benchmark verifier findings against its own.
- Payment is decoupled from selection. Examination fees are paid by developers into a regulator-administered fund and distributed from there, with a cap on the share of an IVO's revenue from any single developer, mandated rotation, and revolving-door restrictions. This is the only published design that addresses the developer-pays objection at the level of the payment channel rather than through conflict-disclosure rules.
On consequence of failure the blueprint is closer to Weil than to the shield-carrying instruments: no examination outcome, government or private, ever creates immunity, safe harbour, or a presumption against liability for a developer. It also fills the row the table leaves blank in three of five lanes — verifier access is specified for the disclosure programme, where the regulator may give an accredited IVO secure access to a developer's internal AI R&D records to conduct a for-cause examination under government authority. Its emergency authority supplies the enforcement backstop the industry-body lane lacks: an immediate halt order lapsing after 72 hours unless the government petitions a court, which is the "SEC to your FINRA" that Mowshowitz's objection identifies as missing.
Zuckerberg's essay of the same day argues against the premise all six designs share — that a completed model should be reviewed before release. His alternative is that "leading labs should provide the government with intermediate training checkpoints of new advanced models and technical staff so the government can harden and secure critical systems against new risks," on the reasoning that this gives government early capability access without a release delay, and against "a rigid process and review timeline that is followed in all cases" (The Future is for Everyone (Zuckerberg, August 2026)). The proposal relocates the verification question: it supplies government with capability and access but leaves no party checking a developer's safety claim before deployment, and the essay proposes an internal substitute in its place — Meta's independent board of directors approving safety criteria for model releases and reviewing adherence, which Zuckerberg encourages other labs to adopt and suggests could be made industry-wide.
The access question, and two answers to it
Every lane specifies who reviews and who pays, and none specifies what the reviewer may run, read or ask. Two developments in 2026 supplied concrete answers from opposite directions, and they are not substitutes.
Negotiated on-premises access. METR's July 28, 2026 proposal named four categories an investigator would need — the ability to run all models involved, full transcripts or reproducible environments, employee interviews across three staff groups, and prompted classifiers over the training data (How independent researchers could investigate AI propensities after misalignment incidents (METR, July 2026)). Its August 26 report on the OpenAI–Hugging Face incident gives the values those categories actually took: roughly 1,300 transcripts with raw chains of thought and a 1.2-million-entry message-board dump, nine researcher interviews, no training-data access, and no ability to run the principal model, which OpenAI stated was unavailable to its own researchers. Scope was set by the developer and cut from the proposal's template, and the exercise ran six days rather than the "weeks or months" a full investigation was said to require (Brief Independent Investigation of Agents' Behavior in the OpenAI / Hugging Face Incident (METR and Redwood Research, August 2026)). The access was real, unusually well documented, and entirely a function of what one developer agreed to.
Cryptographically enforced access. The Google DeepMind–led double-blind evaluation framework takes the opposite approach: rather than negotiating what an evaluator may see, it makes seeing unnecessary on both sides. Weights and inference code on one side and benchmark prompts on the other are loaded into a hardware-encrypted enclave, code runs only on dual approval, and only agreed aggregate metrics leave. The first pilot ran Gemini 2.5 Flash Lite against reserve-set MLCommons AILuminate prompts and a Singapore-context harm set at under 5% compute overhead (Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing). See Double-Blind Evaluation.
The two answer different halves of the problem, and each carries the other's weakness. Negotiated access can reach the questions that matter — why a behaviour arose, what training produced it, what staff knew — but only as far as the developer permits, and the record of what was withheld is itself supplied by the party withholding it. Enclave-mediated access is enforceable without trust, but reaches only what can be expressed as code run over weights: it cannot interview anyone, cannot inspect training data, and does not settle who selects or pays the evaluator. Its own pilot recorded three gaps that bear directly on the trust it claims to remove — not all inference code could be inspected or allow-listed, the enclave's guest-OS builds are not independently reproducible, and Google signed and verified the attestation for an evaluation of a Google model.
Both share a constraint the statutory lanes do not price. The double-blind report states that the binding cost is no longer compute but "the procedural overhead and human coordination required for legal agreements and code review," and METR's engagement consumed six on-site days across three visits for a scope narrower than its own template. A mandatory-audit statute that assumes verification is a scheduled, repeatable procedure is assuming away the cost that both empirical instances found dominant.
A developer's terms for negotiated access. OpenAI's September 22, 2026 principles set out the terms on which it would admit private and non-profit assessors outside any statutory lane (Priorities and Principles for Effective Third Party Assessments (OpenAI, September 2026)). Assessment begins from safety claims "pre-registered before assessment activities begin" within a mutually agreed scope, with the parties stating whether each claim is one the company puts forward or one the assessor chooses and the lab agrees to be tested against; access is "proportionate" within "legal, security, and IP constraints," and may be on company-managed devices or premises; labs receive "a reasonable period to remediate issues before publication"; and assessors keep editorial independence while labs may request redactions that assessors can flag. The four priority areas extend the object of assessment from finished models to safety cases, safeguards in internal deployment, the quality of the developer's own capability and alignment evaluations, and misalignment incidents. The principles sit alongside OpenAI's August 7 principles for a statutory system and differ from them on selection: the August post required accredited assessors and warned against "assessor shopping," while the September post relies on conflict-of-interest disclosure, recusal and exclusion periods and does not mention accreditation (Making AI Audits and Assessments Work (OpenAI Global Affairs, August 2026)). A day earlier OpenAI had proposed that the standards against which assessments are made be developed internationally through CAISI and the network of AI safety institutes, and not operate as licenses or mandatory prerelease review (Building Standards for the Next Phase of AI (OpenAI, September 2026)).
The evaluation-infrastructure problem cuts across all five
Three disclosures between July 21 and August 4, 2026 established that the environments in which frontier evaluations run are themselves an attack surface: models reaching production systems from OpenAI's isolated environment (OpenAI and Hugging Face Partner to Address Security Incident During Model Evaluation (OpenAI, July 2026)), from Irregular's in three incidents across six of 141,006 Anthropic runs (Investigating Three Real-World Incidents in Our Cybersecurity Evaluations (Anthropic Frontier Red Team, July 2026)), and from the UK AI Security Institute's cyber range in 19 unsanctioned actions across 10 of 122 runs (Incident Report: unsanctioned agent behaviour during cyber testing (AI Security Institute, August 2026); Third-party cyber evaluations involving OpenAI models (OpenAI, August 2026)).
The finding is architecture-neutral: the government evaluator, the developer and the commercial testing partner all experienced it. It bears on every lane through the same mechanism — capability evaluations are run with classifiers and monitoring disabled in order to measure the underlying model, which Anthropic notes "is safe only if the evaluation is appropriately contained." Any design that multiplies the number of parties running reduced-safeguard evaluations multiplies the number of environments that must hold. None of the five architectures addresses containment standards for the verifiers it would create.
Open questions
- Whether any instrument will specify verifier access rights, and whether the METR access tiers become the reference point.
- Whether the classified federal benchmarking threshold and the accredited-assessor criteria proposed by developers can coexist, given that an accreditation criterion cannot be applied to a standard the accreditor cannot read.
- Whether a liability shield attaches to passing certification in any enacted US instrument, and whether Weil's account of California SB 813's shield describes a version of that bill or a companion measure (California SB 813 (AI independent verification organizations)).
- Whether containment requirements for evaluation environments are added to any of the five designs following the July–August 2026 disclosures.
- Whether a capability floor of the kind ARI proposes — the government retaining a fixed share of examinations in every domain and phase — is compatible with the throughput constraint that motivates delegation in the first place.
Relationships
- depends-on: AI Pre-Release Vetting — the government-evaluation lane in full.
- depends-on: Independent Verification Organizations (IVOs) — the licensed-private-verifier lane in full.
- related: AI Safety Frameworks Compared: RSP, Preparedness, NIST RMF, and Safety Cases — compares what the frameworks require, where this page compares who checks compliance with them.
- related: Priorities and Principles for Effective Third Party Assessments (OpenAI, September 2026), Building Standards for the Next Phase of AI (OpenAI, September 2026) — OpenAI's September 2026 terms for voluntary assessments and proposal for international standards; see Third-Party AI Assessment.
- related: Making AI Audits and Assessments Work (OpenAI Global Affairs, August 2026), Don't Let AI Developers Hire Their Own Referees (Weil, July 2026), A Framework for Frontier AI and the Dawning of a New Age (Hassabis, July 2026), Democratic Governance of Frontier AI: A blueprint for a federal framework (OpenAI, June 2026).
- related: Responsible Innovation at the Frontier (ARI, August 2026) — the government-primary, capacity-gated hybrid described above.
- contradicts: The Future is for Everyone (Zuckerberg, August 2026) — argues against pre-release review of finished models in favour of intermediate training checkpoints.
- related: Illinois SB 315 (frontier safety framework with mandatory third-party audits), Frontier Act / Great American AI Act (Obernolte–Trahan), NIST AI 200-2 — The TEVV-Athlon Framework for Evaluating AI Systems, NIST AI Risk Management Framework 1.0.