Autonomous weapons, also called lethal autonomous weapons systems (LAWS), are weapons systems that can identify, track, and engage targets with no human in the loop at any point in the decision-to-kill chain. They are a frontier AI governance issue and the trigger for the Anthropic–Department of War conflict. AI systems are increasingly capable of autonomous targeting, but a bright line between "AI-assisted" and "fully autonomous" weapons is difficult to draw, and no international treaty specifically governs them. The principal governance instruments and positions in the debate are US Department of Defense (DoD) Directive 3000.09, the International Committee of the Red Cross (ICRC) position calling for a binding treaty, and the operational deployment of AI targeting through Project Maven.
Amodei lists AI-enabled autonomous weapons as a component of the AI and authoritarianism risk — particularly the risk that autocracies use autonomous weapons for internal repression and external aggression without the human moral friction that has historically constrained the use of force.
Project Maven
Project Maven is the US military's AI targeting system, documented in Katrina Manson's book Project Maven (W.W. Norton, 2026), reviewed by the Economist on May 14, 2026 (The AI that transformed American warfare (Economist, May 14 2026) — review of Manson's Project Maven). Manson is a Bloomberg national-security journalist (entity created).
Function
Maven fuses photos, text, radio, and electromagnetic-pulse intelligence into target identification and coordinates response options — which plane, which munitions, closest to which target. A NATO official quoted in Manson called it "the Microsoft Windows of warfighting," and the book describes that "with a single click by a human, Maven can turn data into ash." Per an NGA official, the system identifies and hits 5,000 targets per day. Large language models are now embedded in the targeting flow, producing a 5× speedup.
Operational history
In 2017–18, counter-terror trials ran in Somalia and Afghanistan, where early algorithms labeled clouds as flying school buses and trees as people. In 2018, Google walked out of Maven after employee protests, and Palantir became the central firm in Maven's development. The system was used in the 2019 operation that killed ISIS leader Abu Bakr al-Baghdadi in Syria, in the 2020 drone strike that killed Iran's Qassem Suleimani, and in 2021 to track the Kabul-airfield crowd during the US withdrawal.
Russia's 2022 invasion of Ukraine scaled Maven's use: it fed "points of interest" at industrial scale, with a $1M/month cloud-computing bill. In late 2022–23, more than 1,500 algorithms were whittled down to two dozen for Ukraine. Afghan models achieved 70% accuracy but dropped to 30% in the Philippines amid jungle terrain, and the system struggled with Ukrainian snow and Russian tanks missing turrets. The 70%-to-30% accuracy figure serves as a reference point for the claim that AI is context-specific. As of 2026, Maven still produces 10 false detections per square kilometer assessed in Ukraine, a reference point for operational reliability tolerance at scale.
On-weapon algorithms
Manson documents two systems carrying Maven-developed algorithms, each able to be sent to find and attack targets on its own:
- Goalkeeper — a loitering munition / suicide drone.
- Whiplash — an explosives-laden jet ski, early versions of which were smuggled into Ukraine by the CIA.
Ground systems followed the air and naval platforms. A $114 million US Army program backing semiautonomous breaching vehicles — already deployed by Ukraine's military against Russian fortifications — was detailed in July 2026 reporting, with program lead Sgt. Maj. Corey Wilkens saying future versions could be "completely agentic" (Source: puck.news).
Personnel
- Drew Cukor (entity created) — Marine officer who founded Maven; reprimanded by the Marine Corps for toxic culture. He is quoted: "Let's be able to look ourselves in the mirror and make sure we are careful."
- General Chris Donahue — commander of American land forces in Europe, who pioneered Maven's use in Ukraine. He is quoted: "Ultimately, all this stuff will become automated."
Claude in operational targeting
NYT (2026-04-12) reported that Palantir's Project Maven was deployed with a military version of Claude (Anthropic) embedded in it: "Embedded with a military version of Claude, Maven helped generate thousands of targets in the opening weeks of the Iran campaign." The report describes this as the first documented public claim that a major frontier AI model was used operationally in targeting for active military operations, and characterizes the degree of human oversight at the "left click, right click" confirmation step as minimal. The report stands in tension with Sullivan's Tech High Ground normative framework calling for responsible military AI adoption and "human-machine teams with ethical norms," and with the ICRC's categorical prohibition on autonomous targeting. The claim comes from a single journalistic source (NYT investigative, April 2026) and has not been independently corroborated elsewhere; confidence in this specific claim is medium pending additional sourcing.
US institutional policy: DoD Directive 3000.09 (2023)
The governing US policy instrument is DoD Directive 3000.09, reissued January 25, 2023 by Deputy Secretary Kathleen Hicks, replacing the 2012 version by Ashton Carter (Source: DoD Directive 3000.09 — Autonomy in Weapon Systems (source summary)). It establishes the minimum standard above which the Anthropic–DoW conflict plays out.
The directive defines three categories: an autonomous weapon system, which once activated selects and engages targets without human intervention; a semi-autonomous weapon system, which only engages targets humans have selected (including fire-and-forget); and a human-supervised autonomous system, in which a human can intervene or terminate.
Its central requirement is that systems be designed to allow commanders and operators to exercise "appropriate levels of human judgment over the use of force" — a formulation critics note replaced an earlier framing that included the word "control." Before formal development and again before fielding, covered systems require tri-signature senior-review approval by USD(P), USD(R&E), and VCJCS, supported by a new Autonomous Weapon Systems Working Group; the Deputy Secretary of Defense may waive review for urgent military need.
New in the 2023 reissue, the directive integrates DoD's five AI Ethical Principles (Responsible, Equitable, Traceable, Reliable, Governable) and requires AI-using weapons to be transparent, auditable, and explainable, rapidly reprogrammable, and capable of being disengaged or deactivated if exhibiting unintended behavior. Under DoDD 5025.01, the directive expires or must be reissued by January 25, 2033. Cyber operations systems, unarmed unmanned platforms, unguided munitions, and mines fall outside the directive, a coverage gap. Per DoD's own acknowledgment, no weapon system has been required to undergo senior review since 2012; critics read this as evidence the threshold is easily drawn around, while defenders read it as evidence no system has yet crossed the LAWS line in practice.
Human Rights Watch / Harvard Law IHRC critique
Human Rights Watch, in a February 2023 review (Source: DoD Directive 3000.09 — Autonomy in Weapon Systems (source summary), citing HRW Feb 2023 review), criticized the 2023 update for removing the word "control" in favor of the vaguer "appropriate levels of human judgment"; shifting from mandatory "shall" to permissive "will," and from "ensure" to "sufficient confidence"; and narrowing the "unintended engagement" definition to exclude indirect civilian harm via infrastructure damage. HRW also noted that no senior review had been triggered in a decade. Its proposed remedy is an international legally binding treaty prohibiting autonomous weapons lacking meaningful human control, a ban on anti-personnel AWS, and a government-wide (not just DoD) policy.
International position: ICRC (2021, reinforced 2024)
The International Committee of the Red Cross — custodian of international humanitarian law (IHL) under the Geneva Conventions — issued its formal position on AWS on 12 May 2021 and reinforced it in its March 2024 submission to the UN Secretary-General pursuant to UNGA Resolution 78/241 (adopted 22 December 2023, 152 states in favour) and in President Mirjana Spoljaric's 3 May 2024 statement to the Vienna Conference on Autonomous Weapon Systems 2024: Humanity at the Crossroads (Source: ICRC Position on Autonomous Weapon Systems (source summary); entity: International Committee of the Red Cross (ICRC)). The ICRC calls for a legally binding international instrument negotiated by 2026.
The ICRC defines autonomous weapon systems as:
"Weapon systems that select and apply force to targets without human intervention. After initial activation or launch by a person, an autonomous weapon system self-initiates or triggers a strike in response to information from the environment received through sensors and on the basis of a generalized 'target profile'."
The consequence the ICRC emphasizes is that the user does not choose, or even know, the specific target, timing, or location of the resulting force.
Concerns
The ICRC raises three concerns. The humanitarian concern is that AWS create risks for civilians and combatants and risks of conflict escalation, assessed through an "effects-based approach" grounded in 160 years of documenting weapons' humanitarian effects. The legal (IHL) concern is that AWS challenge users' ability to comply with distinction, proportionality, and precautions in attack, each of which requires contextual human judgment at the time and place of the attack — judgment the ICRC says is impossible when the user does not know the specific target or timing. The ethical concern, which the ICRC presents as independent of IHL compliance, is that "ceding life-and-death decisions to machine sensors and software is a dehumanizing process that undermines our shared humanity" (ICRC 2024 submission; echoed by Spoljaric, Vienna 2024).
Recommendations
The ICRC recommends new legally binding rules comprising two prohibitions and strict regulation of everything else. First, it would prohibit unpredictable AWS — "autonomous weapon systems that are designed or used in a manner such that their effects cannot be sufficiently understood, predicted and explained" — on indiscriminate-effects grounds. In Spoljaric's 2024 framing, this captures "unpredictable autonomous weapons powered by opaque machine learning algorithms," a challenge to the use of modern deep-learning systems in targeting chains. Second, it would prohibit anti-personnel AWS — "autonomous weapon systems that are designed or used to apply force against persons" — on combined IHL and ethical grounds; the ICRC's categorical line is strongest here, holding that even an AWS technically complying with distinction should not make life-and-death decisions about humans.
Third, the ICRC would strictly regulate all remaining AWS (those used against objects rather than persons, and predictable in their effects) through limits on target type (constraining targets to objects that are military objectives by nature); limits on duration, geographical scope, and scale of use; limits on situations of use (for example, environments where civilians or civilian objects are not present); and requirements for human–machine interaction, including effective human supervision, timely intervention, and deactivation capability.
Meaningful human control
The ICRC's operational content of "meaningful human control" — the concept distinguishing its position from DoD 3000.09's "appropriate levels of human judgment" — comprises sufficient information about weapon, target, environment, and context of use; sufficient time to assess the lawfulness and appropriateness of each attack and to intervene if circumstances change; and the ability to act, meaning to activate, supervise, intervene in, and deactivate the weapon. Where these conditions cannot be satisfied, the ICRC regards the use as presumptively unlawful or ethically impermissible.
Contrast with DoD Directive 3000.09
The ICRC position is contradictory to Directive 3000.09 at the structural level: the US takes a national-policy-only, regulation-with-review approach, while the ICRC seeks treaty-level categorical prohibition of defined categories.
| Dimension | DoD Directive 3000.09 (2023) | ICRC Position (2021/2024) |
|---|---|---|
| Form | National military policy | Treaty-level legally binding instrument |
| Approach | Regulate via senior review | Prohibit + regulate |
| Human-control standard | "Appropriate levels of human judgment" (no control over specific target selection required) | "Meaningful human control" (information + time + ability to act) |
| Anti-personnel AWS | Permitted with senior review | Categorically prohibited |
| Unpredictable / opaque-ML AWS | Not separately addressed | Categorically prohibited |
| Scope exclusions | Cyber, unarmed platforms, mines, UXO outside directive | Position applies to all AWS as defined |
| Governance level | DoD only; not government-wide | International law; States as binding subjects |
| Sanctions for noncompliance | Internal administrative | Treaty-based state responsibility |
Treaty track and 2026 target
The ICRC's institutional strategy has shifted from the stalled CCW Group of Governmental Experts — which has discussed LAWS since 2014 without producing binding rules — toward the UN General Assembly First Committee track opened by Resolution 78/241. President Spoljaric's Vienna 2024 framing set 2026 as the target year for negotiating a binding instrument, coincident with Austria-led momentum in UNGA proceedings.
UN Secretary-General António Guterres reinforced the prohibition track on July 6, 2026, calling in a Geneva speech on AI governance for lethal autonomous weapons to be "banned by international law" and describing machines that select and engage targets without human control as "morally repugnant" — remarks that echoed Pope Leo XIV's encyclical (below) and revived the dispute at the core of Anthropic's court fight with the Pentagon over autonomous-weapons and surveillance assurances (Source: wsj.com). See Anthropic v. United States (Pentagon ban challenge).
Religious and normative voice: Pope Leo XIV's Magnifica humanitas (May 2026)
In his first encyclical, "Magnifica Humanitas: On Safeguarding the Human Person in the Time of AI" (released May 25, 2026), Pope Leo XIV addressed lethal autonomous weapons as a religious and normative voice. In coverage dated May 31, 2026, Leo called for the world to "disarm" AI, condemned lethal autonomous weapons, and declared traditional just-war theory "outdated" in the context of machine-mediated lethal force (Source: ncronline.org; villanova.edu).
The encyclical's line on war — "there is no algorithm that can make war morally acceptable" — anchors the autonomous-weapons critique. Leo's just-war argument runs parallel to the ICRC's ethical concern: the just-war tradition presumes human moral agents exercising proportionality and distinction judgments, which AWS that select and engage targets without human intervention cannot perform. The intervention adds a civil-society and religious voice (see Religious and Civil-Society Voices on AI) to the treaty-prohibition side of the debate, alongside the ICRC, HRW, and the Campaign to Stop Killer Robots.
The Anthropic–DoW conflict
A governance dispute involving autonomous weapons plays out above the DoD Directive 3000.09 floor (Source: Clawed). Anthropic's classified DoW contract included a restriction that Claude could not be used to control lethal autonomous weapons — stricter than Directive 3000.09 requires, since the directive permits LAWS with senior review while Anthropic's term categorically excluded LAWS use. The Trump administration accepted these terms, then reversed course, objecting to privately imposed policy limitations on the military. The conflict escalated to the DoW threatening to designate Anthropic a "supply chain risk," a designation normally reserved for foreign adversaries.
In Dean Ball's analysis, the Anthropic restriction was a policy constraint imposed through a contractual vehicle — arguably unwise as a mechanism, since in his view policy should come through legislation or the directive itself rather than defense contracts, but not illegal or unprecedented. Ball argues the DoW's response, corporate destruction rather than contract cancellation, chose the most destructive option among those available rather than the least restrictive means. The directive is the institutional backdrop: it establishes the minimum standard, Anthropic's contract terms raised it, and the conflict is about who sets the policy above the floor.
Defense acquisition push: SCSP AI+ Expo (May 6–9, 2026)
At the Special Competitive Studies Project's AI+ Expo (May 6–9, 2026), a coordinated joint statement from senior defense leaders pressed for greater public- and private-sector spending on interoperable autonomous weapons systems. The signatories were Emil Michael (DOD Under Secretary for Research and Engineering), Mike Duffy (DOD Under Secretary for Acquisition), and Mike Obadal (Army Under Secretary), with European and Pacific defense counterparts joining publicly (Source: insideaipolicy.com).
At the same conference, Eric Schmidt argued that the Pentagon must abandon most major weapons programs and restructure acquisition for autonomous AI-driven warfare: "anything that's exquisite, big, slow and emits heat — it's a problem." (Source: insideaipolicy.com). The same week, DOD Under Secretary Michael delivered a "never again single-threaded" speech (May 7, recorded in earlier dev-log).
Read together, the SCSP signals point to a defense-side argument that the existing major-weapons procurement pipeline is obsolete, that interoperable autonomous systems are the binding constraint on future US warfighting capability, and that the acquisition side is the policy lever (cf. Procurement-Driven AI Governance).
The programs reached a live-fire milestone in July 2026: on July 10, Anduril's YFQ-44A became the first American Collaborative Combat Aircraft to fire an air-to-air weapon, launching an AIM-120 AMRAAM over the Mojave Desert in an end-to-end, beyond-line-of-sight strike against a simulated target, with Anduril's Lattice software ingesting the target track and a human operator tasking the engagement (Source: airandspaceforces.com). Public opposition has followed the products: a coalition including BAYAN Washington planned a July 19, 2026 rally at Anduril's Seattle office over autonomous weapons, citing the company's autonomous naval vessels and RIMPAC participation (AI Backlash) (Source: geekwire.com).
US legislative proposals (2026)
Congressional proposals to mandate human oversight continued alongside the Schiff HALO Act and Gillibrand SAMA Act: on July 17, 2026, Representatives Beyer, Barrett, and Jacobs introduced the Human Authority over Autonomous Weapons Act, a bipartisan House bill requiring human oversight and approval for intentionally lethal autonomous weapons (Source: axios.com).
Governance gap and key tensions
No international treaty specifically governs autonomous weapons. The UN Convention on Certain Conventional Weapons (CCW) has discussed LAWS since 2014 but has not produced binding regulation. As a result, the primary constraints on autonomous weapons development are national military doctrine and rules of engagement (in the US, DoD Directive 3000.09); individual company policies, such as Anthropic's contract terms; informal norms among democratic militaries about human-in-the-loop requirements; and advocacy by the ICRC, HRW, and the Campaign to Stop Killer Robots for treaty-level prohibitions. UNGA Resolution 78/241 (December 2023) and the Vienna 2024 conference signal a possible shift from the stalled CCW process to a UNGA-track negotiation, targeting a binding instrument by 2026 per the ICRC.
Several tensions structure the debate:
- Regulation vs. prohibition — DoD 3000.09 regulates; the ICRC and HRW want categorical prohibitions for defined classes.
- National policy vs. international law — the US approach is national-policy-only, while treaty momentum in the CCW has stalled.
- Definitional elasticity — whether a system is "autonomous" or "semi-autonomous" turns on which targeting functions are automated, making it easy to draw the definition around a given system.
- Private-sector policy vs. institutional floor — Anthropic's categorical LAWS exclusion is stricter than DoD permits, and the DoW responded with corporate coercion rather than accepting the private floor or rewriting the directive.
- Scope exclusions — cyber, unarmed drones, and target-nomination software fall outside the directive, leaving large portions of military AI ungoverned by this instrument.
Relationships
- instance-of: none (this is the parent concept)
- depends-on: DoD Directive 3000.09 — Autonomy in Weapon Systems (source summary) — the institutional US policy baseline
- contradicts: ICRC Position on Autonomous Weapon Systems (source summary) vs. DoD Directive 3000.09 — Autonomy in Weapon Systems (source summary) — treaty-prohibition vs. national-regulation approaches
- related: International Committee of the Red Cross (ICRC) — international-humanitarian-law voice on AWS
- related: ICRC Position on Autonomous Weapon Systems (source summary) — ICRC's 2021/2024 formal position
- related: Clawed — Anthropic-DoW conflict case study
- related: Pope Leo XIV — Magnifica humanitas calls to "disarm" AI, condemns lethal autonomous weapons, declares just-war theory "outdated"
- related: Religious and Civil-Society Voices on AI — religious/normative voice on the treaty-prohibition side of the AWS debate
- related: AI and Authoritarianism — autonomous weapons as tool of authoritarian control
- related: America's AI Action Plan — includes "AI & Autonomous Systems Virtual Proving Ground"
- related: Export Controls (AI) — supply-chain designations against Anthropic undermine export-control framework
- supports: Mutually Automated Destruction: The Escalating Global A.I. Arms Race (NYT) — NYT reporting on Claude embedded in Project Maven for Iran targeting
Related Concepts
- AI and Authoritarianism — autonomous weapons as a tool of authoritarian control
- DoD Directive 3000.09 — the governing US policy
- ICRC Position on Autonomous Weapon Systems — the IHL-side counterweight
- International Committee of the Red Cross — the issuing entity
- Clawed — the Anthropic-DoW conflict as governance case study
- America's AI Action Plan — the AI Action Plan's DoD provisions include an "AI & Autonomous Systems Virtual Proving Ground"
- Export Controls — the weaponization of supply-chain designations against Anthropic undermines the export control framework