Author: Nita Farahany Source: https://nitafarahany.substack.com/p/when-ai-fails-who-pays-inside-my Published: September 30, 2025
This is the eleventh installment (Class 11 of 27) in Nita Farahany's published account of her AI law and policy course. The class examines AI liability: how harms caused by AI systems fit, or fail to fit, within traditional products-liability frameworks, comparing the European Union's layered approach to the United States Durbin-Hawley AI LEAD Act, and surveying the problems that remain even where liability is clear.
The bromism case
The class is anchored on a real case. A 60-year-old man was hospitalized with bromism after consuming sodium bromide in place of table salt for three months on ChatGPT's advice; his blood bromide level reached 1,700 mg/L, around 200 times normal. Farahany uses the case to frame AI causation as a puzzle. The causal chain ran from ChatGPT suggesting bromide, to the patient buying it, to a retailer selling it (sodium bromide is mostly used to clean hot tubs), to the patient consuming it over months, to hospitalization. The class asks who the "cheapest cost avoider" is along that chain: OpenAI, the retailer, or the patient.
Products-liability theories applied to AI
The class applies three traditional products-liability theories to AI systems. A manufacturing-defect theory addresses a specific unit that was made wrong; Farahany treats it as inapplicable to AI because every instance is identical. A design-defect theory addresses a whole product line designed badly, judged by a reasonable-alternative-design test; it is applicable here, raising the question whether ChatGPT should be designed not to suggest bromide. A failure-to-warn theory, analogized to medication that does not warn about drowsiness, is also applicable, raising the question whether ChatGPT should have warned not to ingest sodium bromide.
Drawing on Buiten, the class identifies two assumptions traditional liability law makes that AI breaks. The first is that users can assess and understand risks: a sharp blade is visibly dangerous, whereas AI risks are not. The second is that manufacturers maintain control over their products: a toaster stays a toaster, whereas ChatGPT today differs from ChatGPT last month. Farahany illustrates the control problem with a continuously-learning car analogy. A traditional car moves from factory to a fixed product the owner controls; an AI car moves from factory to a learning system shaped by millions of drivers into an evolved system, leaving open who controls it.
The EU "3-layer cake"
The class describes the European Union's approach as a three-layer arrangement. The first layer is the AI Act, passed in 2024, which sets pre-market safety standards. The second is the Revised Product Liability Directive, also passed in 2024, under which software is now treated as a "product" and a developer that keeps updating it remains liable. The third would have been the AI Liability Directive, proposed in 2022 and withdrawn in 2025, which would have created presumptions of causality and forced evidence-sharing; after three years of negotiation in which the parties could not agree on basics, it was withdrawn. Farahany labels the three layers, in her class's framing, "vanilla," "mint chocolate chip," and "Tide-Pod-flavored," respectively.
The Durbin-Hawley AI LEAD Act
The class presents the bipartisan Durbin-Hawley AI LEAD Act, summarized by its sponsors with the line: "When a defective toy car breaks and injures a child, parents can sue the maker. Why should AI be treated any differently?" Section 101 establishes liability for AI developers for defective design, failure to warn, express warranty, and unreasonably dangerous claims. Section 102 provides that a party that uses an AI system and "substantially modifies" or "intentionally misuses" it becomes liable as if it had created the system. Section 301 authorizes civil action by the U.S. Attorney General, a state attorney general, an individual, or a class. Farahany flags as an open question what counts as "substantial modification" — for example, whether a hospital fine-tuning ChatGPT for medical advice would incur liability.
The remedies problem
Drawing on a framing attributed to Cathy Sharkey, the class argues that liability problems persist even where liability is clear. A recall problem arises because there is no clear way to "recall" ChatGPT when millions depend on it. Damages are described as insufficient because money paid after harm does not prevent the harm. Injunctions are described as disruptive: an order to "stop giving medical advice" could entail banning any health-related query.
Alternative liability mechanisms
The class surveys alternative mechanisms beyond traditional products liability. These include safe harbors tied to standards, on the model of the DMCA and the RISE Act of 2025; liability caps paired with mandatory insurance, citing Japan's ¥100M robotics cap; and no-fault compensation funds on the model of the vaccine-injury program.
Relationships
- part-of: Nita Farahany intro course series (Class 11 of 27)
- related: AI LEAD Act (S. 2937), AI Liability, Eu Product Liability Directive (planned)
- previous: Inside My AI Law & Policy Class 10: The Glass Box Paradox (Farahany, September 2025) next: Inside My AI Law & Policy Class 12: Red-Teaming AI (Farahany, October 2025)