Don't Let AI Developers Hire Their Own Referees (Weil, July 2026)

"Don't Let AI Developers Hire Their Own Referees" is a guest commentary by Gabriel Weil published in AI Frontiers on July 29, 2026, in its Policy & Regulation section. Its published subtitle states the argument: "Letting AI developers pick their own safety auditors creates a conflict of interest. Requiring liability insurance instead would put insurers' own capital behind risk assessments."

Weil is an assistant professor at the University of Houston Law Center and a Non-Resident Senior Fellow at the Institute for Law & AI, serving on the board of Principles of Intelligence. His research concerns the role of liability in mitigating catastrophic AI risk; he previously worked on climate policy for the Climate Leadership Council and the White House Council on Environmental Quality.

The piece advances a position rather than reporting a finding, and is treated as such throughout. It is also the most complete single survey of the live independent verification organization instruments held here, each with its primary-source citation.

The target: the IVO family

Weil describes a "growing chorus of scholars and policymakers" favoring private rather than governmental governance of frontier AI, in which "the state sets the safety outcomes it wants and licenses independent verification organizations (IVOs) that compete to certify developers against those outcomes." He names three variants: Gillian Hadfield's "regulatory markets," under which developers pay for oversight from private regulators that governments license and hold accountable (Regulatory Markets: The Future of AI Governance); Dean Ball's "private governance," which Ball likens to bank supervision (Source: hyperdimensional.co); and the nonprofit Fathom's conversion of the latter into model legislation (Source: fathom.org).

He states the proponents' rationale without dismissing it: legislators and agencies "understand these systems less well than the labs building them," rules fixed in advance cannot keep pace, and private verifiers sit closer to the technology and are disciplined by competition.

The four live instruments

This is the article's citation of the FRONTIER Act press release, which is the canonical primary URL for the bill's introduction. Weil's account of SB 813 and of Virginia does not reconcile with the record held on California SB 813 (AI Standards and Safety Commission) and in Fathom chief executive Andrew Freedman's July 2026 statement that Virginia has already passed an independent-verification bill; both discrepancies are recorded and unresolved on Independent Verification Organizations (IVOs).

The design flaw Weil identifies across all four: "AI developers would typically select and pay the organizations that certify them, giving IVOs a financial incentive that might clash with high safety standards."

The argument

Developer-pays selection favors leniency. Weil argues the model reproduces the conflict of interest that discredited the credit-rating agencies after the 2008 financial crisis: when issuers shopped for the agency that would bless their securities, competing agencies were pressed toward leniency. An IVO dependent on the developers it clears for repeat business "has a significant incentive to grade gently," and a developer shopping among IVOs will find the one that does, so that "competition — the feature that is meant to make the IVO model effective — instead drives it toward laxity." He credits the credit-rating parallel to earlier critics (Source: transformernews.ai). Where certification also carries a liability shield, as under SB 813, he argues the problem compounds: "a shield swaps the broad incentive to cut risk by any cost-effective means for a narrow incentive to do only what earns the shield."

Government licensing reintroduces the problem it was meant to solve. The proponents' answer — a public body licenses IVOs and decertifies lenient ones — requires "a public body with the expertise to second-guess technical judgments and the independence to withstand pressure from powerful firms." If the state could reliably field one, "much of the reason to outsource verification at all would fall away."

Underwriters Laboratories read against its proponents. Weil treats UL, the standard rejoinder, as evidence for his own case. UL began in 1894 as the Underwriters' Electrical Bureau, backed by fire-insurance underwriters "because their capital was on the line when buildings burned," and "earned its authority in the decades when it answered to these underwriters, who paid for its mistakes" (Source: ul.org). The manufacturer-pays arrangement survives today, he argues, only because "consumers purchasing a product are usually the same people the product might harm," so demand for safety is real. Frontier AI breaks that alignment: "much of the risk generated by frontier AI development and deployment falls on nonconsenting third parties" — his example is a model conducting a cyberattack against a company that is neither its developer nor its user — leaving customer demand "too weak a signal to motivate adequate investments in safety."

The auto-insurance comparison. The Insurance Institute for Highway Safety "arguably illustrates a better path for frontier AI regulation than IVOs": because insurers pay liability claims they fund IIHS to rate cars on pedestrian crash prevention, and "IIHS certification stays honest because insurers have money at stake and demand accuracy. The IVO model has no party with that kind of skin in the game. Requiring AI developers to carry liability insurance would create one" (Source: iihs.org).

The mandatory-insurance proposal

Under the proposal the insurer occupies the verifier's chair "but with a much stronger financial incentive to conduct competent risk assessment": if it underprices it pays in claims, if it overprices it loses the client, and "since insurers' capital stays exposed for as long as the policy runs, they keep monitoring the developer and can make continued coverage conditional on fixes to safety issues that arise."

Weil states four properties. No expertise is lost, since an insurer can hire the same specialists or contract the evaluation to a firm "which would answer to a principal that loses money if the assessment is incorrect." Liability's reach extends, because "the largest harms that AI systems might cause greatly exceed the liquidation value of AI companies, and judgments above that value deter nothing." No immunity is conferred: "unlike the safe-harbor versions of the IVO model, mandatory insurance would confer no legal immunity: instead, the developer would carry liability and insure against it." And the net effect is to "convert a potential race to the bottom on safety assessment into a competition to price risk accurately."

Government would still set outcomes and license verifiers, but the tasks become "relatively simple" — set minimum coverage levels for different circumstances and apply standard solvency and conduct rules. "Neither task would call for expert judgment from the government about which AI models are safe." Insurers could impose operational rules through policy terms (Source: ssrn.com).

The insurable layer and its limit

Weil confines the mechanism's claim to what he calls the "insurable layer." Insurers would price on "models' demonstrated capabilities and how they are deployed, drawing on the growing ecosystem of third-party AI evaluators," and for risks reasonably likely to materialize competition forces accuracy, since underpricing is a loss the insurer eats. Tail risks are different: for "risks of extreme but very low-probability catastrophes" the insurer "would have no more at stake than a paid auditor," because the costs "would be too high for the insurer to cover, making them 'judgment-proof.'"

Two legislative asks follow: liability-insurance legislation "should therefore also require insurer-verifiers to disclose their risk assessments and bear liability for false ones, and it should keep targeted public oversight on the risks the premium does not price." He notes disclosure duties are ordinary insurance practice, since "carriers already file their rates and forms with state regulators." His summary: "The insurer is the right verifier for the insurable layer. It is not, by itself, the answer for the most extreme risks."

He adds a general caution against reading risk-bearing as sufficient: insurers "built the inspection regimes behind boiler and pressure-vessel safety" (Source: repository.law.umich.edu) and "long conditioned aviation coverage on airworthiness and pilot qualification," but "risk-bearing is necessary for good verification, but it is not sufficient. Insurers get it right when the design around them requires it, through mandatory coverage with cost-sharing that keeps the insured parties exposed, backed by capital adequate to the losses underwritten."

Four objections answered

Political feasibility. "The FRONTIER Act builds the scaffolding that the insurance model needs: licensed verifiers, federal oversight, and audit duties for the largest developers. Connecting verification to insurance would require only an amendment to pending legislation that already has bipartisan support." He cites drivers, contractors and nuclear operators as precedents for requiring proof of ability to pay, and argues on cost that "an industry that raises tens of billions of dollars for compute can carry premiums proportioned to the risks it creates. If an AI company is forced to slow down or shift direction because it cannot afford the insurance premium demanded for its practices, that is the system working."

Voluntary coverage is insufficient. Existing voluntary activity is named as evidence of feasibility rather than sufficiency: AI companies buying agent coverage (Source: prnewswire.com) and joining efforts to build insurance-linked safety standards (Source: fastcompany.com). But "the market is unlikely to generate coverage for the largest potential catastrophes on its own, since AI developers lack strong incentives to purchase insurance that covers liabilities for which they would otherwise be judgment-proof."

Pricing without loss history. "Underwriting AI liability insurance does not depend on knowing the precise probability of catastrophe. Insurers need only a price they are prepared to stand behind, set conservatively where the evidence is thin." Precedents named: satellite launches insured before any satellite loss history; cyber coverage emerging while loss data accumulated; the 1957 Price-Anderson Act forming nuclear insurance pools "to price reactors that had never melted down." Existing AI-adjacent lines: Munich Re insuring AI model performance since 2018 (Source: munichre.com), Lloyd's underwriters backing chatbot-error and hallucination policies (Source: the-decoder.com), and cyber carriers extending coverage to AI-caused security failures and deepfake-enabled fraud (Source: coalitioninc.com). Available pricing inputs before any loss are "an AI model's training compute, capability and safety evaluations, and deployment scope," and where risk is ambiguous "insurers charge more for the ambiguity" (Source: link.springer.com) — a surcharge Weil describes as operating "as a tax on opacity and misalignment," lowered fastest by making risk legible.

Keeping pace. Policies would cover developers rather than particular models, written year to year like commercial liability coverage, with premiums adjusted as operations change and re-underwriting triggered by "a new frontier training run, a move from closed API to open-weights release, or capabilities beyond what was initially evaluated." Ordinary model updates within the evaluated range are handled by insurer monitoring under the existing policy.

The tail

Weil calls uninsurable tail risk "the most formidable hurdle" and does not claim to solve it: "COVID-19 cost the United States an estimated $16 trillion" (Source: jamanetwork.com), "an AI-enabled catastrophe could cost as much or even more, and no private pool of capital can stand behind that number. This proposal does not pretend otherwise." Some risks are "practically noncompensable: the losses would exceed insurable limits, bankrupt any defendant, or arise in catastrophic scenarios where the legal system could not meaningfully function," so that "even unlimited legal liability imposed after the harm occurs would generate too little incentive to prevent it in the first place."

Two tools are named for that layer. Shared residual liability "would put every frontier developer on the hook for a share of any catastrophe that one of them causes," mirroring "the Price-Anderson Act's retrospective assessments, which reach every licensed nuclear reactor after an accident occurs at any of them," which would "multiply the assets behind a judgment and give each firm a stake in the care its rivals take." A public backstop would follow the Terrorism Risk Insurance Act, under which government shares the cost of extreme harms with insured parties; Weil describes TRIA's cap limiting payouts to $100 billion per year as "a design choice, and Congress can set it to match the peril."

He also restates a standing position held "since my earliest work on AI liability" — "that courts should award punitive damages in near-miss cases, calibrated to ensure that the AI developer internalizes the risk of the catastrophe that their practices nearly caused" (Source: ssrn.com).

The closing move argues the two layers are not separable in practice: "the case for insurer-verifiers does not depend on solving the tail," because the precautions that reduce insurable losses — "tighter security against model theft, stronger containment during evaluation and deployment, and closer monitoring of what agents actually do" — are largely the ones that reduce the uninsurable downside. His illustration is the July 2026 intrusion: "The failures it revealed — weakened safeguards and breached containment — are the same ones that the gravest scenarios would run through. And the verification infrastructure that would be built to evaluate insurable risks is exactly what any approach to the tail would also need" (OpenAI and Hugging Face Partner to Address Security Incident During Model Evaluation (OpenAI, July 2026)).

Position in the debate

The argument runs against the third-party-audit architecture that federal and state instruments have converged on — Frontier Act / Great American AI Act (Obernolte–Trahan), California SB 53 and New York RAISE Act — and against Freedman's separate reading of the Supreme Court's June 2026 Slaughter decision as strengthening the case for accredited IVOs reporting to CAISI (Trump v. Slaughter). Weil's concluding formulation grants the proponents their premise while denying their mechanism: "The private governance movement has the right instinct. The government is poorly positioned to certify the safety of frontier AI models, and a competitive market of expert verifiers could do better. But competition yields accuracy only when there is a cost to being wrong."

Provenance

Captured from the publisher's canonical URL on July 30, 2026. Publisher citation metadata (author, publication date, journal title) matches the byline and dateline, and the on-domain author page matches the claimed affiliation. Three inline chart images and an embedded audio-narration player were not captured; the published "Footnotes" heading rendered with no footnote content.

Relationships

• contradicts: Independent Verification Organizations (IVOs) — argues the developer-pays structure drives the model toward laxity
• contradicts: California SB 813 (AI Standards and Safety Commission) — the certification-for-tort-shield design is the version Weil argues compounds the problem
• related: Frontier Act / Great American AI Act (Obernolte–Trahan) — the instrument Weil would amend rather than replace; this source supplies its canonical press-release URL
• supports: AI and Tort Liability — mandatory insurance as a liability-preserving alternative to certification safe harbors
• related: AI Liability — the liability regime an IVO shield would displace
• related: Gillian K. Hadfield, Dean Ball — the proponents whose framings the piece takes as its target
• related: Institute for Law & AI — the author's affiliation
• related: Trump v. Slaughter — the removal-power ruling read in the opposite direction by Fathom
• related: OpenAI and Hugging Face Partner to Address Security Incident During Model Evaluation (OpenAI, July 2026) — the incident used to argue the insurable and uninsurable layers share precautions