Author: Nita Farahany Source: https://nitafarahany.substack.com/p/why-governing-ai-synthetic-media Published: October 21, 2025
Class 15a of a 27-part AI law and policy course by Nita Farahany, published October 21, 2025. The class examines the governance of AI-generated synthetic media. It uses a hypothetical political candidate, "Sarah," whose campaign is damaged by a viral deepfake, to trace the six lifecycle stages of the DHS mitigation framework, arguing that at each stage the available interventions either fail on effectiveness, run into First Amendment scrutiny, or both. The class is anchored on a September 2025 OpenAI result that AI hallucinations have irreducible mathematical lower bounds (Source: https://nitafarahany.substack.com/p/why-governing-ai-synthetic-media).
Summary of argument
Farahany distinguishes hallucinations from deepfakes by intent: hallucinations are unintentional errors, which she presents as mathematically inevitable, while deepfakes are deliberate fabrications. Most regulation, the class argues, targets the intent behind synthetic media rather than its falsehood as such. The class is framed by two anchoring examples cited in the source description: the October 15, 2025 imposition of a C$5,000 fine on Quebec litigant Jean Laprade for filing AI-hallucinated citations, and OpenAI's September 4, 2025 mathematical treatment of hallucination as unavoidable.
On the OpenAI result, the class reports that the generative error rate is bounded below by roughly twice the "Is-It-Valid" (IIV) misclassification rate, so that even with 95% identification accuracy a model retains about a 10% generation error rate. Farahany attributes the irreducibility to three factors: rare information in training data, model approximation, and computational intractability. Better engineering reduces the error but does not eliminate it.
The six-stage walkthrough
The class applies the six stages of the DHS lifecycle mitigation framework to Sarah's hypothetical political deepfake.
Stage 1 — Intent (criminal deterrence). The Take It Down Act reaches "intimate visual depictions" only and so does not cover Sarah's political deepfake; California SB-926 is similarly limited, while Hawaii SB 2687 might apply. The class identifies a constitutional constraint: California AB 2839 was struck down by Judge Mendez in October 2024 as a content-based restriction that was a "hammer instead of a scalpel," with counter-speech framed as the constitutional remedy for false speech.
Stage 2 — Research (organizational preparedness). The class cites the engineering firm Arup's loss of $25 million to a deepfake impersonation of its CFO, and notes that even one-hour detection still leaves roughly 100,000 views.
Stage 3 — Creating the model. EU AI Act Articles 50(2) and 50(4) require both provider and deployer marking of synthetic content. China's Administrative Measures, effective September 2025, require clear visible labels plus technical identifiers. Denmark and the Netherlands provide 50-to-70-year IP-style protection over a person's likeness; the class cites Hugenholtz's critique that this commodifies persona and applies the wrong frame.
Stage 4 — Dissemination (platform duties). The Take It Down Act sets a 48-hour removal requirement. California AB 2655 was struck down by Judge Mendez on Section 230 grounds. France's Bill 675 imposes €50,000 platform fines. The class observes that by the time platforms remove content at hour 48, roughly 2 million views may already have occurred.
Stage 5 — Detection and disclosure. The class surveys disclosure-labeling laws: Utah SB 131 (audio disclosure at the start and end plus visual labels), Indiana HB 1133 (campaign communications), Michigan HB 5141, New York S 9678, Colorado HB24 1147, and EU AI Act Article 50(4). It cites NIFLA v. Becerra (2018) on compelled-speech scrutiny, framing the tension that effective disclosures must be prominent enough to work, while prominent disclosures may unconstitutionally burden expression.
Stage 6 — Victim remedies. The Take It Down Act again reaches intimate imagery only. Other cited measures include Idaho HB 664 (FAIR) and Florida CS/HB 919. Defamation law is available but slow. The class lays out a court timeline: an emergency injunction in 3 days, an expedited hearing in 3 weeks, a normal hearing in 3 months, and a full trial plus appeals in 3 years; Sarah needs relief in 3 weeks, which the system cannot deliver.
The class concludes that across all six stages, no stage could have prevented Sarah's harm at the timescale required. In the hypothetical she loses the election by 6 points, with 32% of voters citing the deepfake.
Key claims
The class closes on what Farahany calls the Brandeis remedy, quoting: "If there be time to expose through discussion the falsehood and fallacies, to avert the evil by the processes of education, the remedy to be applied is more speech, not enforced silence." Farahany endorses more speech rather than less as the constitutional answer, while acknowledging it may be insufficient and that Sarah may lose anyway.
Relationships
- part-of: Nita Farahany intro course series (Class 15a of 27)
- related: Synthetic Media / Deepfakes, TAKE IT DOWN Act, Section 230 (47 U.S.C. § 230) (planned), Hallucinations Mathematical Inevitability (planned)
- previous: Inside My AI Law & Policy Class 14: When Anyone Can Fake Anything (Farahany, October 2025) next: Inside My AI Law & Policy Class 15: When AI Learns to Manipulate (Farahany, October 2025)