Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security (Chesney & Citron, California Law Review, 2019)

"Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security" is a law-review article by Bobby Chesney and Danielle Citron, published at 107 California Law Review 1753 (2019). It sets out to provide what the authors describe as "the first in-depth assessment of the causes and consequences" of deep-fake technology, surveys the technology's benefits and harms, and evaluates technological, legal, administrative, coercive and market responses. The article introduces the term liar's dividend for the benefit that liars gain, as public awareness of deep fakes grows, from being able to dismiss authentic audio and video as fake (Liar's Dividend).

Publication and provenance

The article appeared in volume 107, issue 6 (December 2019), pages 1753–1820, with DOI 10.15779/Z38RV0D15J. It circulated beforehand as SSRN working paper 3213954, dated July 14, 2018, posted July 21, 2018 and last revised December 17, 2019 (Source: papers.ssrn.com). The published byline gives the authors as Bobby Chesney, James Baker Chair at the University of Texas School of Law and co-founder of Lawfare, and Danielle Citron, Professor of Law at Boston University School of Law and Vice President of the Cyber Civil Rights Initiative. Chesney is the first-named author in both the published article and the SSRN listing, and the publication year is 2019; 2018 is the date of the working paper. The acknowledgments record presentations at, among others, the University of Maryland School of Law's Trust and Truth Decay symposium and Northeastern's 2019 symposium on AI, Media, and the Threat to Democracy.

The article addresses manipulated audio and video produced with machine-learning techniques, principally neural networks and generative adversarial networks (GANs). It predates the text-to-image diffusion models and large language models that later citations of it address, which is the sense in which subsequent work such as Persily's Digitalist Papers essay extends its argument to a different generation of tools.

Summary of argument

Introduction

The article opens with the 2018 episode in which a still image and video of Parkland student Emma González ripping up a sheet displaying a bullseye target were altered to show her tearing up the US Constitution. The authors attribute the limited harm in that case to three factors: the genuine image was already widely available, national attention led journalists to debunk the fakes, and the fakes were of poor quality. They argue that "the situation is about to get much worse" as the technology matures and diffuses.

The label "deep fake" arose from pornographers' use of the technology to place celebrities' faces into sex videos; the authors use it "more broadly, as shorthand for the full range of hyper-realistic digital falsification of images, video, and audio." They state that they are not the first to observe that deep fakes will spread beyond pornography, and claim instead to provide the first comprehensive survey of the harms and possible responses, including early warning of "the powerful incentives that deep fakes produce for privacy-destructive solutions."

Part I: Technological foundations

The authors describe neural networks trained on examples to categorize and generate audio, video and images, citing a University of Washington tool that altered video of Barack Obama so that he appeared to say things he had not. They identify generative adversarial networks, "invented by Google researcher Ian Goodfellow," as the approach likely to produce increasingly convincing fakes, and cite audio synthesis work including DeepMind's WaveNet, Baidu's DeepVoice and the startup Lyrebird. They infer that state actors are conducting classified research in the area but state that it is unclear whether it leads or lags commercial and academic work.

On diffusion, the authors draw on Benjamin Wittes and Gabriella Blum's argument that technologies spread unless a scarce tangible input serves as a chokepoint. Because deep-fake capability depends on knowledge rather than scarce materials, they conclude it "is sure to diffuse rapidly no matter what efforts are made to safeguard it." They cite the FakeApp desktop tool, released after a Reddit user posted face-swapping code, and a New York Times reporter who used it with 1,861 images of himself and 1,023 images of actor Chris Pratt.

Three dynamics, in the authors' account, make deep fakes prone to spreading: information cascades, in which people pass along what others appear to believe; the human tendency to share negative and novel information, illustrated by a study of 126,000 news stories shared on Twitter from 2006 to 2010 that found false rumors reached people ten times faster than accurate stories and were 70 percent more likely to be retweeted; and filter bubbles reinforced by platform algorithms.

Part II.A: Beneficial uses

The authors identify three categories of beneficial use. In education, deep fakes could produce videos of historical figures addressing students or alter film scenes to illustrate hypotheticals. In art, they cite the digital recreation of the late Peter Cushing in Rogue One and the use of recordings to extend Carrie Fisher's dialogue in The Last Jedi, as well as satire, parody and activism. Under autonomy, they cite the prospect of restoring the voices of people with conditions such as ALS and of consensual "avatar" experiences.

Part II.B.1: Harms to individuals and organizations

Under exploitation, the authors describe extortion and "sextortion," fraudulent kidnapping claims backed by fabricated video or audio, and deep-fake sex videos. They cite a now-closed subreddit of deep-fake sex videos of female celebrities with more than 100,000 users and a Discord user who described making a video of a former classmate from about 380 photos taken from her social-media accounts. They argue that most victims of fake sex videos will be women, as with cyber stalking and non-consensual pornography, and quote Mary Anne Franks: "If you were the worst misogynist in the world, this technology would allow you to accomplish whatever you wanted."

Under sabotage, they describe reputational attacks across employment, romance, sport, business and politics. They cite a 2009 Microsoft study reporting that more than 90 percent of employers use search results in hiring decisions and that in more than 77 percent of cases those results have a negative effect, and give the example of a fake released at the moment of a professional sports draft, where later debunking cannot undo the loss of draft position.

Part II.B.2: Harms to society

The authors list scenarios including fake videos of officials taking bribes, soldiers shown killing civilians, a fabricated police shooting, an audio clip "revealing" criminal behavior by a candidate on the eve of an election, and emergency officials "announcing" a missile strike or pandemic. They then discuss nine categories of societal harm:

• Democratic discourse: Deep fakes will let individuals "live in their own subjective realities, where beliefs can be supported by manufactured 'facts.'"
• Elections: The authors introduce the idea of decisional chokepoints, "narrow windows of time during which irrevocable decisions are made," and use the 2017 Russian operation against Emmanuel Macron's campaign, which combined document theft, alteration and a poorly timed dump, as an illustration of what a better-executed operation with deep-fake video or audio might achieve.
• Trust in institutions, public and private, from agencies and courts to religious institutions and advocacy organizations.
• Social divisions, citing Russian activity in 2016 including conspiracy theories about the Jade Helm military exercise and competing Houston protests organized by Russian operators.
• Public safety, citing the 2018 false ballistic-missile alert in Hawaii and Internet Research Agency campaigns simulating a chemical disaster in Louisiana and an Ebola outbreak in Atlanta.
• Diplomacy, citing the injection of fake quotations of Qatar's emir into a Qatari news site in the Saudi–Qatari crisis.
• National security, including wartime deception, claims about civilian casualties in occupied areas, and operations designed to create pressure to restrict an intelligence agency's capabilities.
• Journalism, where news organizations may be chilled from reporting real events for fear the evidence is fake, and may be targeted with deep-fake "stings."
• The liar's dividend (next section).

The liar's dividend

Section II.B.2.i, "The Liar's Dividend: Beware the Cry of Deep-Fake News," addresses a harm the authors describe as "different in kind" from the others: lies that take the form of denials. They identify two mechanisms. In the first, a person accused of wrongdoing fabricates altered audio or video that appears to contradict the accusation, a strategy they describe as high-risk except where resource inequality makes exposure unlikely. In the second, "liars aiming to dodge responsibility for their real words and actions will become more credible as the public becomes more educated about the threats posed by deep fakes," because "a skeptical public will be primed to doubt the authenticity of real audio and video evidence."

The authors name the effect at the start of the following paragraph: "Hence what we call the liar's dividend: this dividend flows, perversely, in proportion to success in educating the public about the dangers of deep fakes." They connect it to mounting distrust of traditional news sources, stoked in their account by President Trump and the "fake news" mantra, and predict that "fake news" will extend to "deep-fake news." Citing the Edelman Trust Barometer's nine-point fall in the US Trust Index from 2017 to 2018, they argue that the combination of "truth decay and trust decay" creates space for authoritarianism, and that cognitive bias will give motivated reasoners "a convenient excuse" to reject true information. The same concern recurs in their discussion of platform policy, where they warn of "bad faith exploitation" of deep-fake awareness through "a well-timed allegation of fakery."

Part III.A: Technological solutions

The authors conclude that detection is losing the arms race. They quote Hany Farid, developer of PhotoDNA: "We're decades away from having forensic technology that . . . [could] conclusively tell a real from a fake." They note that DARPA had begun a contest pitting GAN generation against detection algorithms, with a program manager skeptical of detection, and that provenance systems then under development were product-specific and would be of limited use until one became ubiquitous enough for dominant platforms to require it for posting. They conclude "it would be foolish to trust that technology will deliver a debunking solution that is scalable and reliable enough."

Part III.B: Legal solutions

No general ban

The authors argue that a flat ban would be undesirable, because digital manipulation is not inherently harmful and a ban would chill parody and political speech and invite politicized enforcement, and likely unconstitutional. Relying on New York Times v. Sullivan and United States v. Alvarez (2012), in which the Court held that "falsity alone" does not remove speech from First Amendment protection, they read the case law as precluding a sweeping ban while leaving room for "carefully tailored prohibitions" of deep fakes falling within unprotected categories: defamation of private persons, fraud, true threats, incitement, and speech integral to crimes such as extortion. They cite Helen Norton's view that laws against impersonating government officials are largely uncontroversial under the First Amendment.

Civil liability

The authors identify threshold obstacles: attribution (anonymizing tools such as Tor may make creators untraceable), creators or platforms outside US jurisdiction, litigation cost against judgment-proof defendants, and the "Streisand Effect" of suing. Against creators, they rate defamation, false light and intentional infliction of emotional distress as the strongest claims, copyright and right of publicity as more limited, and the public-disclosure and intrusion-on-seclusion torts as a poor fit.

Platform liability and Section 230

The authors argue that platforms may be "the most efficient and effective" point of intervention and that Section 230 of the Communications Decency Act, as interpreted by courts, has "evolved into a super-immunity." They recount its origin in the Prodigy decision and the aims of Representatives Chris Cox and Ron Wyden and Senators James Exon and Slade Gorton, contrast the "good faith" language of § 230(c)(2) with the unqualified § 230(c)(1), and quote Rebecca Tushnet's description of "power without responsibility." They discuss the 2018 Allow States and Victims to Fight Online Sex Trafficking Act (FOSTA), which added an exception to Section 230 immunity for knowingly assisting, supporting or facilitating sex-trafficking offenses, and the controversy over it.

Building on Citron's earlier work with Benjamin Wittes, they propose that § 230(c)(1) protection "be conditional rather than automatic," available only to platforms that show they have taken "reasonable steps" to ensure they are not being used for illegal ends; platforms that fail the standard could be treated as publishers of user content. To address over-deterrence and private censorship, they suggest four safeguards: a sunset provision paired with data-gathering on removals and automated filtering; damages caps; a federal anti-SLAPP provision; and an exhaustion requirement under which plaintiffs must first notify the platform and allow it a period to respond.

Criminal liability

Criminal law avoids the judgment-proof problem and brings investigative capacity, but the authors describe law enforcement's record on online abuse as "lackluster." They identify the federal cyberstalking statute, 18 U.S.C. § 2261A (up to five years' imprisonment and fines up to $250,000), state online-impersonation statutes, criminal defamation, and 18 U.S.C. § 2101 on incitement to riot, the last constrained by Brandenburg's imminence requirement. They note that state laws against election lies have been struck down, citing Brown v. Hartlage, but suggest that laws against impersonating officials or candidates may survive; they expect criminal law to matter little against foreign intelligence services.

Part III.C: Administrative agencies

The authors assess the Federal Trade Commission, Federal Communications Commission and Federal Election Commission and find each role limited. The FTC and state attorneys general could act against deep fakes in advertising, but efforts against fake news more broadly would meet Section 230, jurisdictional doubts expressed by former Bureau of Consumer Protection director David Vladeck, and First Amendment concerns. The FCC appears to lack jurisdiction over social-media content, having reversed its 2015 reclassification of internet service providers in late 2017. The FEC does not regulate the truth of campaign statements, but the authors see room for a requirement that deep-fake creators disclose that content is fake, noting that major online advertising platforms were then outside FEC disclosure rules and that the rules do not reach issue ads.

Part III.D: Coercive responses

For foreign sources beyond judicial reach, the authors discuss military responses in armed conflict (network operations, kinetic strikes subject to the law of armed conflict, capture), covert action (which requires no armed conflict and whose statutory framework is silent on international law), and sanctions under the International Emergency Economic Powers Act (IEEPA). They trace the April 2015 national emergency on malicious cyber-enabled activities (Executive Order 13694; the article's body text prints the number as 13964, its footnote 273 as 13694), its December 2016 amendment to cover election interference, and the Treasury Department's March 2018 sanctions on Russia's Internet Research Agency, whose stated basis they describe as muddying whether information operations without hacking qualify. They recommend a new national emergency "specifically tailored to attempts by foreign entities to inject false information into America's political dialogue," without any need to show hacking.

Part III.E: Market solutions

The authors predict "a profitable new service: immutable life logs or authentication trails" that would let a deep-fake victim produce a certified alibi, built on wearables, encryption and blockchain-based record-keeping and on partnerships with platforms. They "urge caution," arguing that such services risk the "unraveling of privacy" described by Scott Peppet, would create what Paul Ohm called a "database of ruin," and would raise questions under Carpenter v. United States (2018) about warrantless government access.

They describe platform terms-of-service agreements as "the single most important documents governing digital speech in today's world" and the likely primary battleground. They note Twitter's impersonation ban and Google's non-consensual pornography policy, and argue platforms should commit to "technological due process" (a term from Citron's earlier work): transparent and specific definitions, exclusion of satire, parody, art and education from bans, notice and an opportunity to challenge removals, and human review rather than exclusive reliance on automated filtering, except for re-uploads of content already determined with due care to be fraudulent.

Conclusion

The authors conclude that "our legal and policy architectures are not optimally designed to respond," that their recommendations "would help with that to some degree, but the problem to a considerable degree would still remain," and that "a great deal of further creative thinking is needed."

Proposals

Relation to later developments

Several of the responses the article discusses were later enacted in forms it did not propose. Federal law now addresses non-consensual intimate imagery, including AI-generated imagery, through a notice-and-takedown duty under the TAKE IT DOWN Act rather than through an amendment to Section 230, and state legislatures have enacted election-deepfake statutes whose prohibition variants have met the First Amendment objections the article anticipated (State Deepfake Statutes (MN, WA, TX, CA)). The liar's dividend is cited in later synthetic-media policy writing, including Persily's Digitalist Papers essay, the Future of Privacy Forum's synthetic content report and Nita Farahany's course materials.

Relationships

• supports: Liar's Dividend — origin of the term (section II.B.2.i).
• supports: Synthetic Media / Deepfakes — early legal survey of harms and responses for manipulated audio and video.
• supports: Section 230 (47 U.S.C. § 230) — proposal to condition § 230(c)(1) immunity on reasonable steps.
• related: State Deepfake Statutes (MN, WA, TX, CA) — anticipates the First Amendment limits on election-lie and deep-fake prohibitions.
• related: TAKE IT DOWN Act — later federal response to the sexual-deep-fake harms the article describes.
• related: AI in Elections and Democratic Institutions — decisional-chokepoint and election-manipulation analysis.
• related: AI and Misinformation, AI Content Provenance, AI and the First Amendment, AI and Content Moderation.
• related: Federal Trade Commission (FTC), Federal Communications Commission (FCC), DARPA (Defense Advanced Research Projects Agency) — agencies whose roles the article assesses.
• related: Robert Chesney, Danielle Citron — authors; Benjamin Wittes — co-author of the Section 230 proposal it builds on; Lawfare.
• related: The Digitalist Papers (Stanford, Volumes 1–2), Synthetic Content: Exploring the Risks, Technical Approaches, and Regulatory Responses, Inside My AI Law & Policy Class 14: When Anyone Can Fake Anything (Farahany, October 2025) — later works using the liar's dividend.