Fiduciary AI is the argument, advanced by Nita Farahany in Your Doctor Has a Fiduciary Duty to You. ChatGPT Doesn't. (Farahany, January 2026) (January 2026), that consumer AI chatbots which function as authoritative advisors in domains such as medicine, law, and finance occupy a legal grey zone because they exercise the influence of an advisor while disclaiming any duty of care to their users. The core asymmetry Farahany identifies is that a doctor has a fiduciary duty, a legal obligation to act in the patient's best interest with confidentiality and the professional standard of care, while tools such as ChatGPT Health and Claude for Healthcare carry only terms-of-service disclaimers.
Background
Farahany situates the argument against a decline in trust in public health institutions. She cites a KFF poll showing trust in the CDC dropping from 66% to 54% in less than a year, with declining confidence among Democrats and longstanding skepticism among Republicans. As confidence in established institutions falls, she argues, people turn to other sources of advice.
The argument is anchored in two product launches. ChatGPT Health (OpenAI) launched January 7, 2026, timed to the JP Morgan Healthcare Conference, and Claude for Healthcare (Anthropic) launched days later. Farahany cites OpenAI's figure that more than 230 million health-related questions are asked on ChatGPT each week, over 40 million per day, often outside normal healthcare hours.
The argument builds on Farahany's earlier work on Cognitive Liberty.
Trust migration
Farahany describes a shift in where people seek health guidance, from after-hours pediatrician hotlines and CDC websites toward uploading a photo of a child's rash to ChatGPT at 2 a.m. and asking whether to drive to the emergency room. She cites a NEJM AI study finding that people prefer AI medical responses to those written by doctors and rate even low-accuracy AI answers as valid, trustworthy, and satisfactory, which she connects to acceptance of harmful advice and unnecessary medical attention.
What distinguishes consumer chatbots from prior AI health tools
Farahany attributes the asymmetry to a combination of four properties that, together, cause these chatbots to function as health authorities while remaining legally non-authoritative:
- Fluency — natural-language interaction indistinguishable from human consultation.
- Personalization — uploaded medical records, wellness app integrations, and health history.
- Always-on availability — accessible at 2 a.m., outside healthcare hours, and far from physician oversight.
- Apparent authority — extensive medical knowledge, citation-style outputs, and professional-sounding phrasing.
The legal grey zone
Farahany describes several positions that, in combination, leave consumer chatbot health advice outside existing legal protections:
- OpenAI: treats HIPAA as inapplicable on the basis that it is not a "covered entity" under the act.
- Anthropic: offers "HIPAA-ready" products, with Claude for Healthcare aimed at HIPAA-compliant organizations using Claude for Enterprise. Farahany argues these protections likely do not apply to consumer use of Claude for Health by individuals uploading their own lab results.
- FDA: has clarified that many AI-enabled software tools fall outside medical device regulation, at least where clinicians can "independently review" the recommendations. Farahany notes that chatbot use increasingly occurs far from clinic walls and physician oversight.
- Privacy promises: both companies state they will not train foundation models on consumer health data, but Farahany notes these commitments rest on the companies' word and could be changed through revised terms of service.
In Farahany's framing, the gap is that HIPAA governs clinical care and consumer protection laws govern products, while always-available personalized chatbots influencing health decisions sit between the two categories. As she puts it:
We have HIPAA for clinical care. We have consumer protection laws for products. These intimate, personalized, always-available AI chatbots, which are now influencing health-care decision making of millions while disclaiming responsibility for them, fall into a legal grey zone. They aren't healthcare providers, so they have no fiduciary duty to their users.
Debates and positions
Farahany states that the argument is not opposition to AI in healthcare, noting that chatbots can synthesize information, translate jargon, help patients prepare for doctor visits, and broaden access. Her concern is the absence of a corresponding legal obligation:
We are rapidly normalizing the transfer of trust from accountable institutions to systems that explicitly refuse accountability. We need to answer sooner, rather than later, what legal obligations should apply to tools that function as health authorities while claiming they are not one — especially when tens of millions of daily users already treat that product as a health advisor.
The legal innovation Farahany proposes is a category of fiduciary obligation that attaches to AI systems functioning as advisors in domains historically governed by fiduciary duty, including healthcare, legal, and financial advice.
Generalization beyond healthcare
Farahany argues the structure generalizes beyond medicine. AI tools functioning as legal advisors, such as consumer-grade chatbots answering legal questions or reviewing contracts for individuals, face the same asymmetry, as do AI tools functioning as financial advisors. In each case the AI functions as an authority within a field historically governed by fiduciary duty while disclaiming that duty.
Relationships
- introduced-by: Your Doctor Has a Fiduciary Duty to You. ChatGPT Doesn't. (Farahany, January 2026)
- depends-on: Cognitive Liberty (Farahany's foundation)
- related: AI Mental Health and Psychological Harm, AI Liability, Healthcare — AI Deployment, Parasitic AI / Spiral Personas (related dependency-on-AI dynamic)
- instance-of: AI Policy (umbrella)
- regulated-by: FDA (AI software guidance), HHS (HIPAA), state medical boards