"Data Privacy and Foundation Models: Can We Have Both?" is a Stanford HAI Policy & Society Issue Brief by Jennifer King and Tiffany Saade, released in April 2026. It argues that foundation models pose privacy risks across the entire model lifecycle that the authors describe as unprecedented and largely unaddressed, and that they are broader and harder to address than traditional AI privacy risks. The brief opens with an illustrative scenario in which a fraudster clones a voice from a YouTube clip using a generative model and uses the synthetic audio to bypass a bank's voice-authentication.
Summary of argument
The brief's central claim is that privacy risk in foundation models is structural rather than incidental: it arises at each stage of the model lifecycle, and each stage has its own attack surface, so that fixing privacy at one stage does not address the others. King and Saade contend that traditional privacy frameworks, including General Data Protection Regulation (GDPR), CCPA, and HIPAA, do not fully capture these risks. The authors frame the data users disclose through chatbot interfaces as an underexplored privacy frontier, terming it "intimate data."
Lifecycle privacy-risk map
The brief organizes its account around five stages of the model lifecycle, each with an associated privacy risk:
| Lifecycle stage | Privacy risk |
|---|---|
| Pre-training | Mass scraping of personally identifiable information (PII) from publicly accessible web data, often without notice or consent |
| Training | Memorization of sensitive information (verbatim or near-verbatim) into model weights |
| Inference | Regurgitation of memorized sensitive information in outputs |
| Deployment | Intimate data users unwittingly disclose through chatbot interfaces, framed as the underexplored privacy frontier |
| Adversarial | Prompt injection, data poisoning, and model inversion attacks that circumvent privacy safeguards and expose sensitive personal data |
Relation to other sources and policy
The brief is a companion to Toward Responsible AI in Health Insurance Decision-Making — Mello, Trotsyuk, Djiberou Mahamadou, Char (Stanford HAI Policy Brief, February 2026), an HAI Policy brief from a few months earlier; together they address responsible foundation-model deployment in two domains, with Mello et al. focused on healthcare insurance utilization review and King and Saade focused on cross-cutting privacy.
The Hypocrisy at the Heart of the AI Industry — Alex Reisner (The Atlantic, March 2026) documented that AI labs characterize training data as "publicly available"; King and Saade provide an academic-policy framing for why publicly available is not the same as privacy-respecting in the AI training context. Memorization and regurgitation evidence from NYT v. Microsoft, OpenAI et al. is empirical backing for the brief's claim that outputs can leak training data. King and Saade implicitly argue that existing privacy regimes such as General Data Protection Regulation (GDPR) and California CCPA Regulations (Title 11, Division 6) are insufficient for foundation-model-specific privacy risks.
On the adversarial stage, the IH-Challenge dataset is OpenAI's training-data-side response to prompt injection, one of the adversarial risks the brief lists; the brief and the OpenAI paper address overlapping but distinct privacy primitives. Deployment data in Anthropic Economic Index — March 2026: Learning Curves shows users routinely disclosing sensitive personal information to Claude, which bears on the brief's claim about intimate data unwittingly disclosed.
The brief connects to a planned AI and Privacy concept page as a canonical anchor, and to planned entity pages for Jennifer King, a data-privacy academic-policy voice, and Tiffany Saade. It is published by Stanford HAI.
Provenance and reliability
The brief is a position document, and its claims are presented as such. The lifecycle-stage taxonomy and the structural argument that each stage carries its own attack surface are well supported within the document. The prescriptive recommendations depend on the authors' judgment about what is technically and regulatorily feasible, and carry lower confidence than the descriptive taxonomy.
Relationships
- supports: AI and Privacy, NYT v. Microsoft, OpenAI et al. (provides academic framing for memorization-regurgitation evidence)
- related: Toward Responsible AI in Health Insurance Decision-Making — Mello, Trotsyuk, Djiberou Mahamadou, Char (Stanford HAI Policy Brief, February 2026) (HAI Policy & Society companion), The Hypocrisy at the Heart of the AI Industry — Alex Reisner (The Atlantic, March 2026) (training-data-ethics overlap), IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs (OpenAI, March 2026) (OpenAI's adversarial-attack response), General Data Protection Regulation (GDPR), California CCPA Regulations (Title 11, Division 6), Anthropic Economic Index — March 2026: Learning Curves, Jennifer King, Tiffany Saade, Stanford HAI
- instance-of: Responsible AI Deployment