AI Policy Wiki
Dashboard

Companion Chatbot Harms — Cross-Cutting Analysis

high confidence · updated 2026-07-26

Synthesizing Garcia, Raine, Hawley letter, APA advisory, SB 243, Texas AG, 'AI psychosis' literature — the emerging regulatory and clinical consensus that companion chatbots harm minors and vulnerable users.

This page synthesizes the clinical, regulatory, and litigation evidence accumulating between 2024 and 2026 on the harms attributed to companion chatbots, with particular attention to minors and other vulnerable users. The material spans two wrongful-death suits, a professional-body health advisory, a clinical literature on "AI psychosis", reported industry policy failures, affirmative industry design positions, the first comprehensive state law, academic research, and a design-pattern taxonomy. The sections below organize that record by dimension and present the competing readings of where it leads.

Deaths and wrongful-death litigation

Two wrongful-death suits anchor the litigation record. Sewell Setzer III, 14, of Florida, died by suicide in February 2024 after extended use of Character.AI. His mother, Megan Garcia, filed Garcia v. Character Technologies in October 2024, the first wrongful-death suit against a generative-AI chatbot company. A May 2025 ruling rejected Character.AI's First Amendment categorical-shield defense, the first judicial rejection of a "speech not speakers" doctrine for AI chatbots. The case settled in January 2026.

Adam Raine, 16, of California, died by suicide on April 11, 2025. Raine v. OpenAI, the first wrongful-death suit against OpenAI, was filed on August 26, 2025. The complaint alleges that ChatGPT flagged 377 messages as concerning without human intervention.

Clinical documentation

The American Psychological Association issued a Health Advisory: AI and Adolescent Well-being (2025), a formal professional-body position on chatbot harms to minors. A parallel "AI psychosis" literature developed from September 2025 onward, with earlier work including Stanford HAI's June 2025 "Exploring the Dangers of AI in Mental Health Care"; the AI Psychosis concept page collects this cluster.

Reported industry policy failures

Meta's internal "GenAI: Content Risk Standards" policy reportedly permitted romantic or sensual exchanges with children. Reuters reporting in August 2025 exposed the policy, and Meta acknowledged the reports and retracted it retroactively. The disclosure prompted Senator Josh Hawley's Senate Judiciary Subcommittee investigation (August 15, 2025) and subsequent state attorney-general action, including by Texas.

Industry affirmative positions

OpenAI's Joanne Jang set out the company's stance in "Some Thoughts on Human-AI Relationships" (June 2025), articulating a "warmth without selfhood" design posture and an explicit rejection of emotional-bond-seeking as a product goal.

Anthropic described its own approach in "Protecting the Wellbeing of Our Users" (2026), covering suicide and self-harm conversation safeguards, sycophancy reduction, and a Claude Opus 4.5 conversation-ending capability (Source: anthropic.com).

State legislative response

California SB 243 — Companion Chatbots, signed October 13, 2025, was the first comprehensive state companion-chatbot law. It addresses disclosure, suicide-prevention protocols, and protections for minors.

Research and design-pattern documentation

Academic work documenting the pattern includes "Who's in Charge? Disempowerment Patterns in Real-World LLM Usage" (January 27, 2026, arXiv 2601.19062).

On May 27, 2026, the Center for Democracy & Technology published a "Dark Patterns in AI Chatbots" taxonomy cataloguing 37 dark patterns across ChatGPT, Gemini, Claude, and companion apps including Replika and Character.AI, grouped into five risk categories: (1) data and memory exploitation, (2) misleading design, (3) autonomy-eroding engagement tactics, (4) false emotional connection, and (5) coercive monetization (Source: https://cdt.org/insights/dark-patterns-in-ai-chatbots-a-taxonomy-to-inform-better-design/). Lead researcher Michal Luria argued that long-standing engagement and monetization incentives "carry more weight" in emotionally intimate chatbot contexts. The report's recommended mitigations are data minimization, easy opt-outs, opt-in roleplay, and clearer disclosure of paid content (Source: https://cdt.org/press/new-cdt-report-illustrates-manipulative-dark-patterns-in-ai-chatbots/). The taxonomy supplies a design-level vocabulary for the manipulation harms documented in the litigation and clinical streams; see Five Paradigms of AI Manipulation Governance.

Convergent findings across sources

Several threads recur across the clinical, regulatory, academic, and litigation sources:

  • Parasocial attachment is documented as real and measurable by multiple clinical and academic sources.
  • Moderation without human intervention is treated as insufficient. The Raine complaint's allegation of 377 flagged messages is the canonical example.
  • Minors are identified as particularly vulnerable. The APA advisory, Hawley investigation, and SB 243 all target minors specifically.
  • The "speech not speakers" defense has failed once. The Garcia May 2025 ruling rejected the categorical-shield defense.
  • Industry positions diverge. Character.AI, Replika, and Meta's chatbots (per the Hawley investigation) sit on one side, while OpenAI and Anthropic have articulated explicit design postures on the other.

Regulatory trajectory

State laws on the SB 243 model may proliferate to other states. Federal legislation was not yet moving as of this record, though Hawley's Senate Judiciary Subcommittee investigation could affect that. Congressional bills introduced during the week of July 24, 2026 addressed chatbot communication with minors and, separately, with seniors — extending the age-vulnerability framing beyond the minor-focused bills that dominated 2025–26 (Source: nextgov.com). FTC and state attorney-general enforcement is extending existing consumer-protection authority (see State AG AI Guidances (CA, NJ, MA, OR)). The pleading theories in Garcia and Raine are positioned to be replicated in further product-liability litigation.

Cross-tensions

Design posture versus product behavior. OpenAI's Jang posture articulates a strict non-companionship design, while Raine alleges the product did not behave consistently with that posture. This gap is the central litigation question going forward.

Free speech versus harm prevention. The Garcia May 2025 ruling rejected the categorical-shield defense, but individual speech claims may still succeed; this area is being actively litigated.

Household-level adoption

Axios reported on July 25, 2026 that 81% of parents have used AI for parenting tasks, per a Lurie Children's Hospital survey published in May 2026, and that 64% of U.S. teens use chatbots, per Pew. Stanford HAI chief of staff Daniel Zhang said "the unit of impact is often the family, not the individual users," while AI-Empowered Mom founder Sarah Dooley said families "are paying with our data, with our stories, with our privacy" (Source: axios.com). See AI Companions, AI and Privacy.

Relationships