Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact is a measurement study by Haofei Xu, Umar Iqbal and Jacob M. Montgomery of Washington University in St. Louis, posted to arXiv on 13 May 2026 (arXiv:2605.14021, cs.CY). It is the first published study to characterise AI Overview activation, source selection, claim fidelity and publisher economic exposure together across a naturalistic query sample.
The paper is an arXiv preprint; nothing in the record read here establishes that it has completed peer review, and its findings should be attributed to the authors rather than treated as settled properties of the system.
Method
The authors built a Puppeteer-based crawler running as AWS Lambda functions in us-east-1, spawning a stateless browser profile per query so that history did not influence results. They took queries from the US-localised Google Trends dashboard across 19 predefined topical categories every 24 hours, issued each to Google, and detected the AI Overview by anchoring on the 24×24-pixel "AI Overview" icon rather than a fixed CSS selector, expanding lazy-loaded content before extraction. Cited reference pages were scraped with readability-lxml, with a headful authenticated retry for failures; short-form social and video platforms were excluded from scraping because of authentication and bot detection. CAPTCHAs were not bypassed; blocked requests were re-queued up to 20 times.
Advertising was measured by parsing each cited page's DOM for <iframe> elements matching EasyList filter rules, and Google's own sponsored ads were captured from the results page and classified as above or below the AI Overview.
Claim fidelity was measured by decomposing each response into atomic claims and checking each against its cited source, using Grok 4.1 Fast Reasoning at temperature 0 with few-shot prompts, on a five-label scheme adapted from privacy-policy consistency analysis: Clear, Vague, Ambiguous, Incorrect, Omitted. Clear and Vague are grouped as consistent; the other three as inconsistent.
Findings
Activation
Across 13 March – 21 April 2026 the crawler issued 55,393 trending queries, of which 7,583 (13.7%) returned an AI Overview. The daily rate showed no monotonic trend but spiked around high-interest public events — the Oscars and NCAA tournament selection on 16 March, the Artemis II launch on 2 April, and the NCAA Round of 64 on 20 March — which the authors note means exposure concentrates where information demand is highest.
Activation varied by a factor of 13 across categories, from 3.5% in Beauty & Fashion to 46.1% in Hobbies & Leisure. Politics (7.5%) and Law & Government (9.6%) fell below the mean, which the authors read as consistent with Google's stated caution on sensitive topics, while Health (26.6%) shows the suppression is not applied uniformly to sensitive areas. Sports supplied 51.4% of the query corpus at a 9.8% activation rate.
Query form mattered more than topic. Question-form queries — those led by one of fifteen interrogatives — activated at 64.7% (2,716 of 4,198) against 9.5% (4,867 of 51,195) for non-questions, a 6.8-fold difference. Among interrogatives, how reached 84.3% and why 73.4%, while who reached 47.9% and did 39.8%. Length amplified activation independently: restricting to non-question queries, the rate rose from 9.9% at one word to 38.7% at six or more.
Source selection
The 7,583 overviews cited 61,212 reference URLs across 7,479 unique hostnames, a median of 8 references each (mean 8.1, range 1–32). The five most-cited hosts were youtube.com (5.49%), en.wikipedia.org (4.39%), facebook.com (3.68%), instagram.com (3.65%) and usatoday.com (2.80%).
Two findings run against expectations. First, cited domains were more credible than the co-displayed first-page results, not less: mean credibility 0.732 across 37,020 scored AI Overview references against 0.645 across 159,752 matched first-page URLs, a gap of +0.087 on a [0,1] scale (95% CI [0.085, 0.089]). Per-category tests reached significance in 14 of 19 categories, all favouring the overview references, with no significant reversal. This contradicts earlier work reporting lower-quality sourcing.
Second, 29.8% of cited domains did not appear in the first-page results at all, which the authors read as evidence of "a source pool and prioritization system distinct from [Google's] own ranking algorithm." The citation distribution is also broader than the results page: 56.3% of the overview's unique hosts were cited exactly once over the window, against 42.1% singleton hosts on the first-page results, despite the results pool holding more than twice as many unique hosts (15,394 against 7,479).
Claim fidelity
Decomposing 7,491 verifiable overviews into 98,020 atomic claims:
| Label | Claims | Share |
|---|---|---|
| Clear | 82,933 | 84.6% |
| Vague | 4,271 | 4.4% |
| Ambiguous | 1,367 | 1.4% |
| Incorrect | 2,609 | 2.7% |
| Omitted | 6,840 | 7.0% |
| Consistent (Clear + Vague) | 87,204 | 89.0% |
| Inconsistent (Ambiguous + Incorrect + Omitted) | 10,816 | 11.0% |
Omission rather than contradiction is the dominant failure mode: 7.0% of claims were not supported by retrievable source text against 2.7% directly contradicted.
The authors identify a measurement artefact in their two lowest-fidelity categories and correct for it themselves. Climate's 48.23% consistency rate is dominated by weather and forecast queries where the overview reports real-time values from Google's structured feeds while the cited pages update continuously, so the verifier reads a stale page and labels a correct claim Omitted or Incorrect; Jobs & Education shows the same pattern on school-closure queries. They therefore describe the 9.64% combined Omitted-plus-Incorrect rate as "a conservative ceiling on substantive AIO unfaithfulness rather than a precise estimate." After excluding the affected subpopulations, 18 of 19 categories sit within an 85.9%–94.8% consistency band, with Health (94.77%), Politics (93.65%) and Science (91.82%) highest.
Activation rate and fidelity are largely independent (p = 0.192), so a low-activation category is not for that reason a safer one.
Publisher economics
Of the 61,212 cited URLs, 30,994 (50.63%) displayed visible advertising. The authors call this a lower bound: a further 14.2% of references point to social and video platforms that the pipeline did not crawl and that were counted as carrying no ads, though those platforms run large advertising businesses.
Exposure concentrates by category: Sports supplied 33.6% of all references, Entertainment 17.2% and Business & Finance 13.0%. The authors caution that low observed ad rates in Health, Law & Government and Travel & Transportation (27–30%) do not indicate safety, since sites in those areas often monetise through subscriptions, paywalls or direct conversions, "and those revenue paths are equally defeated when an AIO answers the query without a click."
Google's own sponsored ads continued to appear on overview-bearing pages, in some cases above the overview. The observed co-occurrence rate was 2.16%, which the authors attribute to a query sample dominated by informational and trending searches rather than commercial-intent ones, and they note the asymmetry "likely carries greater force in the commercial-intent query space our methodology underweights."
The authors' framing and stated limits
The paper opens by noting Google's report of more than 2 billion users encountering AI Overviews, and characterises them as "arguably the most consequential deployment of generative AI in existence: not necessarily the most capable, but by far the most widely and passively encountered," since unlike a chatbot they are "injected into the default search experience."
The concluding section treats unsupported claims as "inherent to generative AI at its current state of development" and not removable by better inputs, and frames the achievable goal as driving down the 11% rate and limiting deployment where errors carry consequences. On economics it describes a feedback loop worth watching, in which sustained traffic displacement reduces the production of the high-quality content that currently grounds the most faithful responses. Remedies it lists — without endorsing one — are revenue-sharing or licensing arrangements, design changes making citations more prominent, and deployment restrictions in categories where the content ecosystem is most fragile, noting proceedings already under way in the United Kingdom, the European Union and the United States.
The authors state what the study does not measure: what users do with overviews, whether errors propagate into belief, whether corrections are sought, and how behaviour varies across user profiles, personalisation states or geographies. They call for audit-ready APIs or structured data-sharing so that this measurement becomes routine rather than requiring "substantial engineering effort to produce a single 40-day snapshot."
Relationships
- related: AI in Journalism and Media — the publisher-referral and advertising findings.
- related: AI and Misinformation — the claim-fidelity measurement.
- related: Google DeepMind — the operator of the measured system.
- supports: LG München I — Google AI Overviews liability (26 O 869/26) — supplies an empirical measure of the click-through displacement at issue.
- related: UK Competition and Markets Authority (CMA), Digital Markets Act (EU) — the source-selection finding bears on the competition theory.
- related: AI Content Saturation (\"AI Slop\") — the degraded-sourcing feedback loop the authors describe.