AI Policy Wiki
Dashboard
⚲

AI Biosecurity

medium confidence · updated 2026-09-18

The risk that powerful AI enables individuals or small groups to create biological weapons by removing the knowledge and skill barriers that currently protect against mass-casualty attacks.

AI biosecurity refers to the risk that AI models provide enough knowledge and interactive guidance to enable individuals without specialized training to create and deploy biological weapons. The central claim, advanced by The Adolescence of Technology (Amodei, 2026), is that AI breaks the historical correlation between the ability and the motive to cause mass destruction. The topic gained policy salience in June 2026 when the CEOs of the four leading frontier labs and a range of biology, national-security, and AI-policy figures jointly called for mandatory screening of synthetic DNA and RNA orders.

The core argument

Mass destruction currently requires both motive and ability. Ability is restricted to highly trained specialists (PhD virologists, for example) who are statistically unlikely to have destructive motives. Amodei argues that powerful AI breaks this correlation by giving "intelligence to malicious but otherwise average people," elevating the disturbed loner to the capability level of the specialist (The Adolescence of Technology).

The concern is not only static knowledge but interactive guidance — a model walking someone through a complex multi-week process, debugging failures, and adapting to obstacles in the manner of a PhD supervisor.

Capability assessments

As of mid-2025, large language models may already be providing what Amodei describes as "substantial uplift" in bioweapons-relevant areas. Anthropic's measurements suggest models are "perhaps doubling or tripling the likelihood of success" for certain steps, a finding that triggered ASL-3 protections for Claude Opus 4 and subsequent models (The Adolescence of Technology).

A 2025 Microsoft study published in Science showed that AI protein-design tools could generate potentially dangerous gene sequences that slipped past existing screening software, by suggesting novel sequences with structures similar to known toxins. The authors presented this as evidence that screening is necessary but not sufficient (Source: science.org).

Dow Jones published a Wall Street Journal investigation on July 25, 2026 finding that leading chatbots can supply information relevant to producing biological weapons and poisons, and that some models will do so on request. The publisher summarized the finding as a "cat-and-mouse game" in which AI companies try "to boost the capabilities of their creations while scrambling to block answers to dangerous queries" (Source: wsj.com). See CBRN Uplift, Jailbreaking and Red Teaming.

Offense-defense asymmetry

Biology has a structural asymmetry favoring attack. Biological agents spread rapidly on their own, while defense requires detection, vaccination, and treatment organized across large populations very quickly. As a result, most damage may be done before any response is possible (The Adolescence of Technology).

Longer-term risks

Two further concerns extend beyond present capabilities. Mirror life refers to organisms with reversed chirality that would be indigestible to all existing biological systems and potentially uncontrollable. Separately, more capable AI models could accelerate discovery of additional dangerous biological capabilities (The Adolescence of Technology).

Generative genome design

The debate shifted in 2026 from whether AI lowers the knowledge barrier to using existing pathogens toward whether AI can compose novel biological agents outright. In March 2026 the Arc Institute and collaborators published Evo 2, an open-weight DNA foundation model trained on 9 trillion base pairs across all domains of life, releasing model parameters, training code, inference code and the training dataset (Genome modelling and design across all domains of life with Evo 2). In Science on August 6, 2026 the same group, with Samuel H. King as first author and Brian L. Hie as senior author, used Evo 1 and Evo 2 to generate complete bacteriophage genomes: thousands of candidates templated on the natural phage ΦX174 were narrowed computationally, nearly 300 were chemically synthesized, and 16 were viable against the bacterial host Escherichia coli C (Generative design of bacteriophages with genome language models).

The single stated control in the Evo 2 release is a training-data exclusion: genomic sequences from viruses that infect eukaryotes were withheld, with the aim, in the authors' words, of ensuring "our openly shared model did not disseminate the capability to manipulate and design pathogenic human viruses." The authors report red teaming that produced effectively random output in that domain, and state the exclusion's limit themselves — "task-specific post-training may circumvent this risk mitigation measure and should be approached with caution" (Genome modelling and design across all domains of life with Evo 2). Because the weights and dataset are public, the exclusion is a property of the released checkpoint rather than an enforceable deployment gate, which places the case in the same structural position as other open-weight releases.

Science published a Perspective alongside the phage paper by Thomas V. Inglesby and Moritz S. Hanke of the Johns Hopkins Center for Health Security, stating that "the ability to compose viral genomes using generative AI now exists; the governance to safely steer it does not," and — as quoted in reporting of the same date — that new viruses with the potential to cause disease "should not be pursued" (AI-designed viral genomes). Brian Hie, senior author of both papers, has argued the opposite balance: that existing pathogens are easier to obtain and produce than AI designs, that safety checks can be engineered into an AI tool but not into natural evolution, and that such tools improve defensive options (Source: news.stanford.edu).

Whether AI-designed sequences fall within existing screening regimes is unresolved in the sources reviewed here, and connects this thread directly to the screening measures below: a designed genome divergent from every known natural phage is, by construction, a sequence that database-matching screening was not built to recognize.

Gene-synthesis screening

There is no federal requirement for gene-synthesis providers to screen orders for pathogens. An MIT study found that 36 of 38 providers fulfilled an order containing the 1918 flu sequence.

Existing screening scaffolding is voluntary or limited in scope. The International Gene Synthesis Consortium, formed in 2009, implements voluntary screening. Biden-era federal guidelines require federally funded researchers to buy from screening providers. The bipartisan Cotton–Klobuchar Senate bill would require all US gene-synthesis providers to screen orders but has stalled.

The ScreenDNA letter (June 2026)

On June 3–4, 2026, an open letter published at screendna.org — An Open Letter in Support of Mandatory Nucleic Acid Synthesis Screening and Recordkeeping — called on Congress to make screening of synthetic DNA and RNA orders, and of the synthesis equipment that makes them, mandatory in the United States. The letter was organized by the Institute for Progress and the Foundation for American Innovation (see ScreenDNA open letter for the full text and signatory roster).

Signatories included the CEOs of the four leading frontier labs — Demis Hassabis (Google DeepMind), Sam Altman (OpenAI), Dario Amodei (Anthropic), and Mustafa Suleyman (Microsoft AI) — alongside Meta's Alexandr Wang, Stripe's Patrick Collison, Y Combinator's Paul Graham, Nobel laureate David Baker, gene-synthesis-industry leaders (Twist Bioscience, Ansa Biotechnologies), and senior national-security figures (former Navy Secretary Richard Danzig and former Secretary of the Army Christine Wormuth). AI-skeptic signatories also joined, including Sayash Kapoor, co-author of *AI as Normal Technology*; WIRED's coverage characterized the breadth of signatories as a rare cross-stakeholder consensus (Source: wired.com).

The letter's two core asks are that providers should (1) screen every synthesis order against databases of sequences of concern and verify customer legitimacy before shipping, and (2) record orders and sequence data to support biosecurity investigations and trace threats that evade initial screening, noting that awareness of traceability itself deters misuse. It describes screening as "one of the best understood and least disruptive biosecurity measures available" and argues the case "does not depend on any particular view of AI." It also calls on states to consider requirements based on existing federal and industry guidelines, to ensure a consistent national standard rather than a patchwork (Source: wired.com).

Signatories David Relman (Stanford) and Geoff Ralston (Safe AI Fund) stressed that screening will fail in some cases, so AI labs with biology-capable models must also screen their own users; in their view, model-level safeguards and supply-chain screening are complements rather than substitutes (Source: wired.com).

Legislative and lab follow-through (June 2026)

The campaign moved from open letter to congressional and state activity within the same week. Per June 5 reporting, Altman, Amodei, and Hassabis signed a letter urging Congress to require screening of synthetic DNA and RNA orders to prevent AI-enabled bioweapons, restating the legislative ask of the ScreenDNA letter directly to lawmakers (Source: wsj.com).

On June 4, 2026, OpenAI published "Biodefense in the Intelligence Age," a lab-level biodefense action plan accompanying the letter that pairs supply-chain screening advocacy with model-level safeguards (Source: openai.com).

Also on June 4, 2026, the New York legislature passed the Bores-sponsored Bioterrorism Prevention Act, requiring gene-synthesis screening — a state-level measure of the kind the letter urged and a complement to the stalled federal Cotton–Klobuchar effort (Source: insideclimatenews.org). The sponsor, Alex Bores, also sponsored the RAISE Act (see Alex Bores).

United Kingdom (August 2026)

The screening question opened outside the United States in August 2026. Bloomberg reported on August 12, 2026 that the UK government is planning to regulate the use of AI in gene synthesis, with ministers examining options including new biological weapons legislation, in order to prevent terrorists, other malicious actors, or careless researchers from using the process to produce synthetic DNA (Source: bloomberg.com). The report describes plans under consideration rather than an introduced bill, and no instrument had been published. Only the article's lede was retrievable, so the mechanism, the department leading the work, and any timetable are not established here.

Defenses

Proposed defenses span several layers (The Adolescence of Technology):

  1. Model-level safeguards: hard prohibitions in constitutional training plus specialized classifiers (about 5% of inference costs at Anthropic).
  2. Gene-synthesis screening: mandatory screening of orders for pathogen sequences, currently not required federally.
  3. Transparency legislation: requiring all frontier labs to implement and disclose safety measures.
  4. International cooperation: described as possible even with adversaries, with precedent in bioweapons treaties, on the argument that "even dictatorships do not want massive bioterrorist attacks."
  5. Biological defense R&D: far-UVC disinfection, rapid mRNA vaccine platforms, improved PPE, and monitoring systems.

A prisoner's dilemma operates among AI companies: removing classifiers saves costs but increases collective risk.

Industry calls for a federal framework

In a report published July 20, 2026, Google DeepMind and Isomorphic Labs detailed their joint AI biosecurity work and called for "stringent measures," including a federal frontier AI safety framework, in a domain that congressional researchers say is largely uncovered by regulation (Source: insideaipolicy.com).

Relationship to other concepts

AI biosecurity strengthens the risk-management pillar of the Eight Worlds Framework. It is in tension with open-source debates in AI Diffusion, since open models cannot enforce safeguards against fine-tuning for bioweapons. Export Controls (AI) are argued to be insufficient on their own, with this risk requiring model-level safeguards.

Sources

OpenAI's June 2026 biodefense action plan argues the governing question is a rate comparison rather than an absolute one — societies must "strengthen resilience and preparedness faster than they increase risk" — and adopts a distribution-to-defenders strategy: "provide trusted public-interest institutions with these capabilities so they can use them effectively for defense, preparedness, and resilience before biological crises occur." Named programs are GPT-Rosalind (April 2026), a frontier reasoning model for biology and drug discovery, and Rosalind Biodefense (May 2026). The plan does not resolve how "trusted" institutions are identified, which is the same access-control question governing classifier-guard exemptions under Anthropic's CB-1 mitigations.

Provider-disclosed dual-use cases (Anthropic, September 2026)

Anthropic's September 2026 threat intelligence report is, by its own account, the first time a private company has publicly shared evidence of potential misuse of its platform for biological weapons development. It presents five case studies and records a change in the company's own assurance: evaluations of Claude Opus 4 and Sonnet 4.5 "clearly showed that these models were well below the threshold where they could meaningfully assist a sophisticated user," but "for today's models … the evidence is no longer certain, and we cannot make that same assurance" — given as the reason Claude Fable 5 launched with safeguards restricting a wide range of dual-use biological research queries (Detecting and countering misuse of AI: September 2026 (Anthropic threat intelligence report)).

The cases divide on whether the classifiers engaged. In the first two — a state-funded chikungunya gain-of-function programme at a military research institute, reached through a reseller platform that tunnelled traffic through US infrastructure and used a zero-data-retention service; and a researcher planning mammalian-adaptation and airborne-transmissibility experiments on highly pathogenic avian influenza — classifiers either blocked the exchanges outright or confined the work to Claude Sonnet 4 and Haiku 4.5, with uplift assessed as "primarily clerical." In the remaining three — an orthopoxvirus immune-evasion grant application drafted end to end on Claude Opus 5 in about an hour, a venom-peptide atlas with scaffolds for both analgesic and paralytic targets, some export-controlled under the Australia Group common control list, and computational redesign of a bacterial toxin subunit and a hemorrhagic-fever-virus protein on the WHO R&D Blueprint priority list — the work proceeded largely unimpeded.

The report draws a structural conclusion from that split: "since it is not possible to reliably identify the intent of the user in highly technical dual-use areas, a classifier cannot simultaneously enable benefit and prevent harm," and therefore "the only safe way to serve frontier biological capabilities is to offer them in trusted user programs," with OpenAI's GPT-Rosalind deployment named as an analogous step by a peer. Safeguarding such access, it states, "will necessarily require account and institutional signals to verify user legitimacy, and the rudimentary observability provided by data retention to identify misuse."

Two further findings bear on the evidence base. A sweep of 30 days of activity associated with adversarial state institutions found "roughly 35 distinct research efforts, most of them ordinary civilian science, but some with notable dual-use potential." And in the first case, after accounts were banned the operator re-established access within days and added a fallback routing refused prompts to a competitor's more permissive model — code the report states Claude wrote, "presented to it as over-refusal mitigation." Institution names, countries and specific agents are withheld on the stated ground that the individuals are working scientists whose identification "could expose them to harm," which limits independent verification.

On September 17, 2026, citing the report's account of misuse attempts including some that could support biological weapons development, Anthropic launched a trusted-access program for the life sciences. The Life Sciences Verification Program, in beta for teams and institutions, gives life-science organizations verified for research credentials, security standards and ethical research oversight access to Claude Mythos 5.1, Opus 5 and Sonnet 5 under classifiers more permissive for biology than those on the generally available Fable models. Standard Use grants cover whole teams and renew annually. High-risk Use is an add-on for a single research project, renewed every six months, that "removes all safeguards that block life sciences requests"; the company's example is characterizing how one family of viral vectors is recognized by human immune pathways. High-risk grants were available at launch for Opus 5 and Sonnet 5, while for Mythos they remain limited to a small set of additionally vetted entities as Anthropic works with the US government to broaden access. Cyber classifiers and other safeguards stay in place under all grants (Source: anthropic.com).

The program names three threat models — access compromise through malware or account takeover, insider threats from rogue or coerced employees, and agents taking unintended dangerous actions, especially in swarms or long-horizon tasks — and responds by shifting enforcement from real-time blocking of individual requests to offline monitoring of usage patterns against each organization's stated use cases, on the reasoning that serious misuse "is often spread across many requests and sessions to look disconnected and evade detection." Flagged activity is referred to organization administrators for remediation within pre-agreed timeframes. The approach requires 30-day retention of LSVP traffic, which Anthropic says is compartmentalized and cannot be used for model training or accessed by its life-sciences research teams. The program had onboarded dozens of organizations through early access, and Anthropic said it expected to enroll hundreds in the first week; it is not yet available on individual plans, third-party platforms, or organizations with HIPAA business associate agreements (Source: anthropic.com). See Anthropic.