AI and privacy concerns the interaction between frontier AI systems and personal-privacy law and norms. The tension spans four axes: the lawfulness of training models on data containing personal information; whether a model's outputs about a person trigger data-subject rights; AI-enabled surveillance; and the privacy model for agentic, computer-use deployments. Running alongside these is a 2026 US debate over whether federal legislation should preempt the patchwork of state privacy laws.
Training-data privacy
The first axis concerns whether AI labs can lawfully train on data containing personal information. In the European Union, enforcement runs through national data-protection authorities, including CNIL (Commission nationale de l'informatique et des libertés) and Garante per la protezione dei dati personali (Italy); the Italian Garante's 2023 suspension of OpenAI remains the template for such actions. Under the GDPR, any processing of personal data requires an identifiable controller and a lawful basis, a framework that web-scale scraping strains at both points. In Canada, the OPC issued a May 2026 PIPEDA finding against OpenAI's training of ChatGPT. At the EU level, the European Data Protection Board adopted a common GDPR data-breach notification template on June 10, 2026 (open for public consultation until August 5, 2026) and, in a plenary with Commissioner Michael McGrath, warned that Digital Omnibus amendments to the definition of personal data would weaken individual data protection (Source: edpb.europa.eu).
In the United States, the question reaches the courts primarily through copyright. The May 2026 Hachette v. Meta suit, which names Common Crawl as a training source, puts training-corpus provenance before US courts, with privacy claims riding alongside the copyright claims.
A distinct strand concerns the status of "publicly available data." A March 2026 report from ITIF maps how diverging national rules on scraping publicly available data are shaping where and how models are trained, and argues for regulation focused on outputs rather than on collection (Source: itif.org). The same question runs through state and national legislation: Connecticut's SB 4 (2026) pairs data-broker registration with bans on the sale of geolocation data, and the UK's Data Use and Access Act 2025 recalibrated allowances for research and scraping.
The data-broker supply route
Scraped web text is not the only path personal data takes into a training corpus. California's data-broker registry began in 2026 requiring brokers to disclose whether they sell data to generative-AI developers, and 32 currently do — the first systematic public record of that supply route in any jurisdiction (Regulating Data Brokers in the Age of AI: A California Case Study).
A Stanford HAI and RegLab audit of all 522 brokers registered with the California Privacy Protection Agency in 2025 found the accountability layer around that route largely inoperative: 45% reported no information about consumer rights requests received, and 9.2% were fully compliant across all four request types required by the California Consumer Privacy Act and the 2023 Delete Act. In a 250-broker subsample of request interfaces, 43% did not support all six consumer rights — a direct CCPA violation — and 22% required a CAPTCHA, one of them eight including math problems. Brokers with less interface friction received more requests. The authors recommend national broker registration, standardised machine-readable reporting, automated cross-broker request mechanisms, a private right of action, and extending privacy protections explicitly to AI development (Regulating Data Brokers in the Age of AI: A California Case Study).
What brokered data is used for once purchased is not established. The brief states there are no transparency requirements mandating detailed disclosure by AI developers, and that little is known about the use.
Inference-time privacy
The second axis concerns whether a model's outputs about a person constitute a privacy event, and whether data-subject rights of access, correction, and deletion extend both to model outputs and to the model weights themselves. Deletion from weights remains technically unresolved, as machine unlearning is not production-grade; data-protection-authority orders that effectively require it therefore function as deployment bans. The OPC PIPEDA finding, together with GDPR Article 15 and Article 17 requests directed at chatbot operators, marks the active frontier of this axis.
A distinct exposure route is share-link indexing rather than model behaviour. After a Reddit user posted on July 25, 2026 that the query site:claude.ai/share surfaced other users' shared Claude conversations and Artifacts, many such pages were found indexed on Google; results were harder to find by the morning of July 26. Google spokesperson Ned Adriance said the pages were indexed across many search engines and that no search engine controls what pages are made public. A comparable incident in September 2025 involved just under 600 conversations by Google's estimate (Source: venturebeat.com; techcrunch.com). The mechanism turns on whether a share feature defaults to a publicly crawlable URL, which is a product-design question rather than a training- or inference-data one.
Subsequent reporting located the failure more precisely and extended it to Bing. WIRED reviewed a sample of the exposed pages over the weekend of July 25–26, 2026 and found they lacked the noindex tag that both Google and Bing say they honour; Adriance said page indexing is Anthropic's responsibility. Google had cleared the shared-chat results by July 27, 2026, though the reviewed pages still lacked the tag. One indexed Artifact contained a clinical trial document listing patients' full names, ages, genders, ethnicities, skin types and treatment dates (Source: wired.com; cybernews.com). The residual exposure is therefore an omitted crawl directive rather than a search-engine behaviour, and de-indexing by one engine does not remove the underlying condition.
A third route is human review of live conversations for model improvement. 404 Media reported on September 14, 2026 that OpenAI is hiring hundreds of contractors, working under the internal codename "Project Lily," who read real ChatGPT users' prompts and rate model responses on a one-to-seven scale. OpenAI told the publication that conversations pass through a Privacy Filter model before contractors see them, while acknowledging that the filter can miss identifiers; the "improve the model for everyone" setting is on by default for free, Plus and Pro users. One reviewer, based in North America and paid more than $50 an hour, was recruited through Crossing Hurdles and paid by Mercor. Anthropic confirmed that it also uses human review and said it removes account identifiers beforehand (Source: 404media.co).
The exposure here is neither a training-corpus question nor a model-behaviour question but a disclosure-and-default one: the data reaching a reviewer is the user's own conversation rather than an inference about them, the filter is acknowledged by its operator to be imperfect, and the consent is a default setting rather than an election. It is the same structure as the share-link cases below in that the product's default determines the exposure, and differs from them in that the recipient is a contracted human rather than a crawler. See RLHF (Reinforcement Learning from Human Feedback), which is the training step this labour feeds.
AI-powered surveillance
The third axis covers AI-enabled face recognition, behavioral monitoring, workplace surveillance, and dragnet biometric identification, covered in depth at AI and Surveillance and AI Bias and Discrimination. The privacy dimension here is the collapse of practical obscurity that follows once recognition and inference become inexpensive.
Camera-enabled wearables
Karianne Tung, Norway's digital minister, said the government is looking to regulate camera-enabled wearable devices such as the smart glasses made by Meta and Snap, in remarks to AFP published August 25, 2026 and reported more widely on September 2. Tung said there is a "clear trend toward combining artificial intelligence with cameras and microphones built into glasses, headphones, hats and other objects we use in everyday life" and that "we must prevent this equipment from being used to monitor other people in public places," adding that measures may include banning facial recognition of other people in public places. She plans to convene a group of experts to advise the government on privacy protection (Source: techcrunch.com).
The framing is distinctive in placing the obligation on the device rather than on the processor of the resulting data: the harm named is bystander surveillance by an individual wearer, for which the consent and data-subject-rights machinery of general privacy law has no natural hook. No bill had been introduced as of September 2026. See AI and Surveillance, Surveillance Technology.
Agentic-AI privacy
The fourth axis concerns agentic, computer-use deployments such as Claude Cowork, OpenAI Operator, and Mariner, which operate inside a user's environment with broad data access, along with enterprise agents that act across email, files, and SaaS systems. This axis is the least developed doctrinally. A March 2026 Frontiers survey catalogs data-leakage and privacy-failure modes specific to agentic systems, including cross-context data movement, memory persistence, tool-call exfiltration, and inter-agent leakage, which have no analog in static-LLM deployments (Source: frontiersin.org). Practitioner analyses raise a controller/processor problem: when an autonomous agent decides what data to access and where to send it, the GDPR's assumption of an identifiable controller making purposive decisions is put under strain (Source: reuters.com).
The peer-reviewed treatment reaches a narrower conclusion than the practitioner framing. Ana Beduschi argues the controller-processor allocation does not break down, because only natural or legal persons can be controllers and agents remain tools deployed by them; the complication is that the controller still fixes the purpose while the agent shapes the means in practice by choosing methods, task sequences and adaptive strategies (Data protection in the era of agentic artificial intelligence (Beduschi)). On her analysis the harder problems are the data-subject rights — access requires reconstructing an evolving decision process rather than explaining one output, portability may reach data about the agent's own operations, and erasure confronts personal data already absorbed into an agent's evolving decision-making. She also proposes a six-level model of agentic autonomy for determining when Article 22(1) applies, concluding it is engaged at every level except full human control. The capability side of these systems is covered at Agentic AI and Agent Autonomy Spectrum (5 Levels).
Agents also appear on the attacker side of data-protection enforcement. Spain's AEPD said on September 14, 2026 that it had received the first notification of a personal-data breach in which the incident was allegedly executed by an AI agent using a well-known language model; the agent logged in, autonomously searched the application for vulnerabilities, and then altered personal data and accessed invoices. The regulator named neither the model nor the organisation and said the notification remains to be analysed. For controllers, processors and data-protection officers it drew four consequences: AI-assisted or AI-executed attacks should be expressly included in processing risk analyses rather than covered by generic references to malware or unauthorised access; response times designed for manual attacks may be insufficient; identities and credentials matter more, since an agent holding an over-permissioned account, API key or token can act at machine speed across services; and processing security cannot rely on manual intervention alone, with human oversight supported by detection and containment able to operate quickly enough (AEPD, first notification of a personal-data breach executed by an AI agent (2026); Source: reuters.com). See AI and Cybersecurity.
US patchwork and federal preemption
More than twenty state comprehensive privacy laws now diverge on definitions, sensitive-data categories, and AI-relevant provisions such as profiling opt-outs and automated-decision-making (ADMT) rules, which, according to a 2026 Foley analysis, makes compliance uniformity practically impossible (Source: foley.com). The federal proposal under consideration is the SECURE Data Act, which would preempt nearly all state privacy laws; its preemption provision split the House Energy and Commerce innovation subcommittee on party lines at its first hearing on June 3, 2026. This privacy-preemption debate parallels the AI-law preemption debate covered at AI Federalism, with a similar structure and similar coalitions.
States continue adding to the patchwork: New Jersey enacted a data broker law (A.5328) on June 30, 2026, requiring commercial data brokers that collect and sell consumer data to register with the state (with an annual $100 fee), as lawmakers advanced additional privacy bills (Source: news.bloomberglaw.com). On July 17, 2026, Gov. Mikie Sherrill's administration said it would suspend enforcement of the law while the legislature amends provisions that threatened political campaigns' access to voter-targeting data (Source: newjerseyglobe.com).
On the law-enforcement side, the US Supreme Court held 6–3 in Chatrie v. United States (June 29, 2026) that use of a geofence warrant to obtain cellphone location data is a Fourth Amendment search (Source: scotusblog.com).
Relationships
- related: AI and Civil Liberties, AI and Surveillance, AI and Democracy, Access Now
- related: AI and Tort Liability (privacy violations as a tort axis), AI Federalism (the preemption fight), SECURE Data Act (House E&C Republican Working Group)
- related: Hachette et al. v. Meta (and Mark Zuckerberg) (training-data axis), AI Copyright Litigation — Analysis
- related: Agentic AI (agentic computer-use privacy), Agent Autonomy Spectrum (5 Levels)
- related: CNIL (Commission nationale de l'informatique et des libertés), Garante per la protezione dei dati personali (Italy), Office of the Privacy Commissioner of Canada (OPC), AEPD (Agencia Española de Protección de Datos) (DPA enforcers); Connecticut SB 4 — Data-broker registration + geolocation-sales ban + facial recognition (CTDPA amendment), Data (Use and Access) Act 2025 (UK), California CCPA/CPRA Regulations — Automated Decision-Making Technology (ADMT)
Sources
Five supporting sources folded by the 2026-06-05 gap scan (ITIF, Frontiers in Computer Science, ScienceDirect, Reuters Practical Law, Foley), plus existing wiki coverage. The page originated as a v4.0 round-3 backlog stub (342 words, sources_count: 0, in-degree 25) and was expanded as the slice-1 thin-anchor action. Foundational ingest candidates: GDPR enforcement decisions, EU AI Act privacy articles, and the ITIF publicly-available-data report.