The LLM fallacy is a cognitive attribution error in which individuals misinterpret LLM-assisted outputs as evidence of their own independent competence, producing a systematic divergence between perceived and actual capability. The term was coined in a 2026 academic paper (arXiv 2604.14807) and cited by Luiza Jarovsky in How AI Is Shaping Us (Jarovsky, April 2026) as a mechanism behind users' reports that AI has made them smarter.
Origin and definition
The paper defines the fallacy as:
A cognitive attribution error in which individuals misinterpret LLM-assisted outputs as evidence of their own independent competence, producing a systematic divergence between perceived and actual capability.
Jarovsky cites the term to explain an observation she draws from social media: many people post that they have become much smarter and that their writing has become much better since they started using AI (How AI Is Shaping Us (Jarovsky, April 2026)). Her response:
Well, maybe it is because the AI system is actually doing the writing.
Mechanism
The fallacy operates as a misattribution of agency: the user perceives the output that appears in their email, document, or chat and mentally credits themselves as the producer. The longer this continues, the wider the gap between what the user can do unaided and what they believe they can do.
Jarovsky frames the fallacy as the demand-side counterpart to the concern raised under cognitive friction. Cognitive friction prescribes deliberate unaided practice to maintain skill; the LLM fallacy explains why users may not naturally seek that practice, because they do not perceive their skill as having declined when the output looks the same or better.
The mechanism sits within a broader documented pattern of AI cognitive effects, including cognitive debt (Kosmyna et al., June 2025 arXiv preprint), skill-formation degradation in junior workers (an arXiv paper Jarovsky cites), and the finding that AI does not reduce work but intensifies it (HBR, February 2026).
Empirical corroboration
The experimental literature on AI-assisted work documents the same perceived-versus-actual divergence the fallacy describes, beyond inference from anecdote.
A 2024 study, "AI Makes You Smarter, But None The Wiser," found that access to AI improves task performance but also inflates self-assessments: participants became less able to distinguish their correct from incorrect answers, so AI raised the score while degrading calibration (Source: arxiv.org). The result states the LLM fallacy as a measurable calibration failure, in which the output improves but the user's metacognitive read of their own ability does not track it.
Reporting on metacognition-and-AI experiments found participants using AI overestimated their own performance by as much as four points on self-assessment relative to actual results (Source: realkm.com). The same reporting describes an inversion of the classic Dunning-Kruger pattern: in AI-assisted contexts, higher self-reported AI literacy has been found to correlate with greater overestimation of competence, so the more fluent users think they are with AI, the more they misattribute the system's output to their own skill (Source: realkm.com).
The HCI literature frames the downstream risk as an "illusion of human competence" that hinders appropriate reliance: users who overrate their own ability may also fail to defer to the AI when they should, and vice versa, so the calibration failure cuts in both directions (ACM CHI 2023, An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems).
Relation to assessment and deployment
The literature points to several consequences for how competence is measured and how products are built. Performance assessments based on AI-augmented output systematically overestimate underlying competence, so decisions about hiring, retention, promotion, and apprenticeship that rely on output-based proxies become less reliable. For individuals, the reliable test of unaided capability is unaided practice, since self-assessment in AI-assisted contexts is not informative. For AI products, those that present output without distinguishing model contribution from user contribution amplify the fallacy, and provenance or attribution interfaces may mitigate it.
Relationships
- introduced-by: arXiv 2604.14807 (2026)
- cited-in: How AI Is Shaping Us (Jarovsky, April 2026)
- related: Cognitive Friction, AI Mental Health and Psychological Harm, Cognitive Debt (related arXiv literature), Parasitic AI / Spiral Personas (related dependency dynamic)
- instance-of: Cognitive Bias