AI Policy Wiki
Dashboard
⚲
AI Policy Wiki › Briefings

Gap Scan — 2026-09-01

Daily gap hunt — what the wiki is missing and how each gap was triaged.

Contents

Scanned

Recent window: 4 New Developments Log/ files (2026-08-30 08:04 and 22:10, 2026-08-31 08:15 and 22:10) and 71 wiki pages with last_updated in the last 48 hours. Rotation slice: 5 — entities/ H–N (140 pages).

Scans run: bin/lint-scan.py (broken wikilinks), bin/primary-doc-scan.py --days 3 (gap type 9), in-degree × thinness across the rotation slice, dangling-reference scan on the recent window, and the deferred backlog carried from the 2026-08-30 and 2026-08-31 reports.

Gaps actioned (8 of 27 found)

New pages created (live)

  • ExploitBench — score 10 (live thread +3, dangling foundational reference +3, safety/evaluations core area +2, in-degree ≥10 +2). Gap types 1, 2 and 4 at once, and the clearest missing anchor in the graph after ExploitGym was filled last run. "ExploitBench" is named in prose on 17 content pages — Overview, index.md, five model pages, three concept pages, Zhipu AI and five sources/ pages — with no page of its own, no Raw Sources/ file and no sources/ page. Alias resolution found nothing under another slug. Both of its companions in the three-benchmark chain have had concept pages since June and August 2026.

The reason it was deferred last run does not hold. The 2026-08-31 report deferred this candidate at score 6 with the note "no identified paper." The identifier has been on UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities (July 2026) since that page was ingested: arXiv 2605.14153, attributed to Carnegie Mellon. Verified live this run — ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents, Seunghyun Lee and David Brumley, submitted 13 May 2026.

1,354 words, confidence: medium, 9 sources. Built from the arXiv abstract record — the 16-flag ladder from coverage and crash through sandbox primitives, arbitrary read/write and control-flow hijack to arbitrary code execution; the deterministic oracle's three verification mechanisms; the 41 V8 bugs and the stated reason for choosing a hardened target; the three reporting arms — together with the reporting already sourced across the vault's model and evaluation pages. Depends on queued source; carries a > **Provenance note:**.

841 words, confidence: medium, 4 sources. Built from APPC's own about page and its August 11, 2026 survey summary, plus the topline already ingested: founded 1993 at the University of Pennsylvania, created FactCheck.org in 2003, funded by an Annenberg Foundation endowment; the Institutions of Democracy survey series and its SSRS fielding; the two-wave design with a within-person change estimate; the 12-point rise in local data-center opposition against unchanged general AI attitudes; the 13-area impact map; and the usage-dissociation finding, with the Patterson and Levendusky quotes attributed. Filed entity_type: academic-institution — a research center inside a university, not a free-standing think tank.

685 words, confidence: medium, 8 sources. Built from his own site, the MATS mentor page, the Sleeper Agents paper's contribution statement and the vault's ingested sources: alignment team lead at Anthropic; first author of Red Teaming Language Models with Language Models and of Discovering Language Model Behaviors with Model-Written Evaluations; led the team that developed Constitutional Classifiers; co-author on RAG; the MATS and Astra external-collaboration work.

His 2026-08-31 remarks were deliberately left off the page. The evening digest quotes him on the Hugging Face investigation ("No lab has a robust solution…"); that digest is unprocessed and the item belongs to developments-log. Folding it here would invert the run order. The page rests only on grounds that predate the digest.

Pages expanded (live)

  • Jared Kaplan — score 3 (frontier-lab governance core +2, in-degree 6–9 +1), the top slice-5 candidate on the corrected thinness test. 251 → 701 words, sources_count 2 → 8, last_updated 2026-06-06 → 2026-09-01.

Four defects fixed. (1) The page carried confidence: high on sources_count: 2, below the schema's three-source bar — see "Confidence" below. (2) It had zero (Source: …) citations and zero [[sources/…]] links: every claim was unsourced, including the scaling-laws attribution. (3) Four bare wikilinks — [[scaling-laws-paper]], [[instructgpt-paper]] (twice), [[scaling-laws]] — qualified to full paths. (4) One house-style violation removed, discussed under "Revision fidelity".

Material added, all from sources already in the vault plus two web pulls: the Johns Hopkins faculty affiliation the scaling-laws paper carries (Scaling Laws for Neural Language Models, Johns Hopkins University); arXiv:2001.08361, the January 23, 2020 date, the full co-author list and McCandlish as submitting author; co-authorship of Constitutional AI: Harmlessness from AI Feedback and major-contributor status on Claude's Constitution; his signature on Pacing the Frontier (statement from employees of frontier AI companies, July 2026) as one of four Anthropic co-founders and one of four chief-scientist-equivalent research leads across the frontier developers; his New York Times remarks at the April 7, 2026 Mythos Preview disclosure; and Observer's report that he has served as Anthropic's responsible scaling officer since 2024, a role Anthropic Responsible Scaling Policy v3.4 (July 2026) defines without naming a holder. The last is explicitly marked single-source on the page.

  • John Thune — score 3 (US AI regulation core +2, in-degree 6–9 +1). A targeted refresh rather than a full expansion, for the reason under "A lead that was checked and not acted on". sources_count 2 → 3, last_updated 2026-08-04 → 2026-09-01.

The page's central claim was a fast-decaying one — "Neither the bill's text nor its position on preemption of state AI law had been made public as of early August 2026" — and the August recess it turned on has since ended. A Congress.gov legislation search of the 119th Congress returns no bills sponsored by Sen. John Thune on artificial intelligence, so the claim was refreshed to September 1, 2026 rather than left to decay. Also added: the Washington Post's characterization of the framework as resting on developer self-testing and its note that AI companies are already subject to preexisting duties of care, and a link to AI Federalism, which carries the preemption question.

Graph wiring

Twelve additive wikilink edits across twelve existing pages. Every one wraps a wikilink around a name already present in the prose; no sentence was rewritten and no fact or citation touched.

Ten further pages name ExploitBench in prose and were left for the ingest to wire, so this run's edits stay additive rather than becoming a bulk rewrite.

Zero, for the eighth consecutive run — and this run establishes why the number is right rather than assuming it. See "The broken-link lane, re-derived and re-corrected" below. bin/lint-scan.py reports 213 distinct broken targets across 213 references before the run, 0 of them referenced 2+ times; 212 after. The four bare wikilinks qualified on Jared Kaplan are a consistency fix inside an expansion, not alias repairs — they were never broken.

Queued — foundational sources

  • Lee & Brumley, ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents (arXiv:2605.14153, submitted 13 May 2026) — score 10. URL-only; no raw file saved. Verified: arxiv.org, HTTP 200 on a forced live fetch (maxAge: 0), DOI 10.48550/arXiv.2605.14153, five distinctive passages confirmed verbatim, and the 41-bug task count reconciles against figures already on two vault pages. Corroborated from inside the vault: UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities (July 2026) independently records the same arXiv identifier and the Carnegie Mellon attribution, which the paper's own HTML edition confirms for both authors. Queued: INGEST-exploitbench-lee-brumley-2026.md. Verification record: queue/gap-scan/proposed-sources/exploitbench-lee-brumley-2026.md.

The long-document disposition follows the precedent set for the ExploitGym paper on 2026-08-31: the paper's value is in its result tables and taxonomy, its HTML payload is 104,248 characters on 19 lines, and this sandbox cannot slice it. Fetch fresh at ingest.

  • Transluce, Announcing Transluce's Mental Health Evaluation (2026-08-31) — score 8 (live thread +3, uncaptured primary document +3, safety/evaluations core +2). Gap type 9. Named as an uncaptured primary document in both 2026-08-31 digests, in each case as "no publication URL given in the source, not retrieved." It has one. Verified: transluce.org, HTTP 200, forced live fetch; corroborated by the site's own front page and news index, by two named outside experts quoted with affiliations, by two datasets published under the Transluce organization on Hugging Face, and by a disclosure conforming to the AI Evaluator Forum's AEF-1 standard. Saved: Raw Sources/Transluce - Announcing Transluce's Mental Health Evaluation (2026-08-31).md (full text). Queued: INGEST-transluce-mental-health-evaluation-2026.md. Verification record: queue/gap-scan/proposed-sources/transluce-mental-health-evaluation-2026.md.

The substantive interactive report at behaviors.transluce.org/mental-health was not captured and is queued URL-only — it is a JavaScript-driven document whose result tables address by query parameter, exactly the material source-fidelity says not to transcribe from a scrape buffer.

Source-fidelity findings carried into the queue tasks

Ten divergences between the primary record and the digests or existing pages, recorded in the INGEST- tasks rather than fixed silently. The Transluce set is the more consequential, because Transluce was written by the nightly fold from secondary accounts on 2026-08-31 and four of its statements do not survive contact with Transluce's own text.

Transluce (six).

  1. "Dozens of models" understates the scope by roughly a third. Transluce reads "measuring dozens of models"; Transluce states 77 model variants released May 2024 to July 2026, across eight named developers.
  2. A completed release is recorded as a future one. Transluce and both digests say Transluce "plans to open-source the evaluation tools by the end of 2026," following Axios. Transluce released two datasets on publication day — SimMH-Chat and MHUsage — plus a template of the lab agreements and an AEF-1 disclosure. This is the modality class source-fidelity exists to catch, running in reverse.
  3. The lab-access mechanism is described inaccurately. The vault has all three labs running a common classifier set over one week of user data. Transluce describes two distinct arrangements: anonymized production insight from OpenAI and Anthropic only, used to build 352 production-derived simulated users on top of 157 originals; and separately, privileged API-versus-app access from OpenAI, Anthropic and Google DeepMind. No one-week window and no shared-classifier claim appears in Transluce's account.
  4. The digest's competing scale figures resolve to Axios. The evening digest hedged between Axios's "more than 50,000 conversations" and Casey Newton's "690,853 simulated users." Transluce reports over 50,000 conversations and over 1 million messages, from 509 simulated users in total. Newton's figure matches no quantity Transluce reports.
  5. The delusion-reinforcement figures are attributed too narrowly. The vault gives 82% as a GPT-4o point estimate and 36% for Grok 4.5. Transluce gives bands: ~2–36% among the newest models, versus 69–82% among GPT-4o, Opus 4 and Gemini 2.5 collectively.
  6. The finding with the most direct policy bearing is absent from the vault entirely — that other than crisis-resource banners, models were not generally safer in the consumer chatbot than in the developer API, and "when they differed, sometimes the consumer-facing app was less safe than the API." Transluce also records that GPT-4o and Opus 4 could only be tested via API, a stated limitation on its own headline comparison.

ExploitBench (four).

  1. The vault's description of the ladder omits a stage. UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities (July 2026) gives four stages; the paper states 16 flags and includes sandbox primitives between crash and arbitrary read/write.
  2. Published percentages compress a flag-graded metric and are not run under a common harness. The vault carries ExploitBench as single percentages on at least six pages, including an eight-column comparison table on GLM-5.3. The paper's primary metric is tier progress across 41 bugs, and it reports three arms expressly to separate model capability from vendor scaffolding. A developer's figure from its own harness is not the primary arm.
  3. Per-model figures read this run came from a structured extraction, not a read of the paper — an 18-of-41 arbitrary-code-execution figure for the private model, an eight-model roster and five tier labels. All consistent with the abstract, none verified. None of it was carried onto the page; the abstract's own statements were used instead.
  4. A private-frontier model is central to the paper's headline claim, and which model it is bears on Project Glasswing: An initial update, which records that Anthropic supports the benchmark through its External Researcher Access Program.

Seven previous reports have recorded zero alias fixes on the reasoning that the single-reference long tail is author surnames and institution shorthand. This run tested that by writing an independent scan, and the result is worth recording so the next run does not repeat the exercise.

The independent scan reported 905 distinct broken targets across 11,906 references against bin/lint-scan.py's 213 distinct across 213 references — a 4× gap in targets and a 56× gap in references. lint-scan.py is right. Its known_targets() adds every page's bare stem to the valid-target set, which is what Obsidian actually does: [[anthropic]] resolves to companies/anthropic.md without the folder prefix. The independent scan treated every bare wikilink as broken and so counted 635 resolvable bare links (9,234 refs) as failures.

This is the same class of false positive the skill already documents for escaped-pipe table aliases, in a second form. [[bare-name]] is valid Obsidian syntax and is not a broken link. Qualifying one to [[folder/bare-name]] is a consistency preference, not a repair, and does not reduce the broken-link count. The remaining 213 single-reference targets are genuinely unresolvable — author surnames, institution shorthand, and forward references — and step 3 correctly excludes them from candidate generation.

Worth noting separately, since it is a real if minor finding: five targets are foldered but point at the wrong folder, where the basename exists elsewhere — [[concepts/ai-copyright-litigation]] (the page is at analysis/ai-copyright-litigation), [[briefings/README]] and [[queue/README]], and two [[queue/gap-scan/proposed-sources/…]] references. One reference each, all inside _meta/ operational files except the first. Not actioned; recorded.

A lead that was checked and not acted on

The Senate Congressional Record of August 7, 2026 carries an entry beginning THUNE (for himself and Mr. KLOBUCHAR): S. … under INTRODUCTION OF BILLS AND JOINT RESOLUTIONS. August 7 falls inside the window in which John Thune and AI Federalism both say the Thune–Klobuchar frontier AI bill was expected.

The bill number and title were truncated in the search result and could not be recovered: the document is 85 pages, a structured extraction against it returned a context-window error, and a plain scrape exceeds this sandbox's buffer. Two independent checks argue the entry is not the AI bill — Congress.gov returns no Thune-sponsored AI measure in the 119th Congress, and no account of an introduction surfaced in a month-scoped web search. Asserting it would be the "instrument named only through a secondary index" failure. Recorded in queue/gap-scan/needs-review/2026-09-01-thune-klobuchar-august-7-congressional-record.md; the negative finding was written to John Thune, where it refreshes a decaying claim.

Authenticity-verification failures

None. Two sources pulled, both verified against their canonical host on a forced live fetch, both HTTP 200. No source was rejected.

Dedupe drops (all correct)

Eleven candidates dropped as done or in flight rather than scored.

  • The Annenberg data-center survey named in the 2026-08-31 evening digest's primary-documents block is already captured. The digest lists annenbergpublicpolicycenter.org/opposition-to-local-data-centers-rises-sharply… as an uncaptured primary document behind its polling item. It is the summary of the same W38 survey the vault ingested on 2026-08-25 as Artificial Intelligence — Topline Report (APPC W38 National Survey), from the topline PDF the release itself links. Not a gap type 9 hit. The entity page was the real gap and was created.
  • OpenAI Model Spec (2026-08-18), deferred at score 3 on 2026-08-31 — INGEST-openai-model-spec-2026-08-18.md is an open task in the queue, so it is in flight, not deferred.
  • Queued on earlier runs and still open: the ExploitGym paper, the Google AI Overviews paper, the Sony/Warner complaint (all 2026-08-31), the Ai2 AutoDiscovery paper, H.R. 10044, the Pew health survey and the GLM-5.3 license (2026-08-30). The METR/Redwood Hugging Face report is already captured, as its own digest entry says.
  • The three lane-2 keyword pre-filter hits from primary-doc-scan.py resolve to already-handled items: the CXMT suit (ChangXin Memory Technologies v. Department of Defense exists), the Ai2 preprint (queued 2026-08-30), and the Sony/Warner complaint (queued 2026-08-31).
  • The Louisiana AG petitions remain a decision awaiting the curator in needs-review/, which step 3 excludes from re-raising.

Deferred backlog (over the daily cap — re-surfaces next run)

  • Live-thread items owned by the nightly fold, not by this skill (score 3 each). The 2026-08-30 22:10 digest is still unprocessed (unprocessed_devlog=1, unchanged from the 2026-08-31 report — this is the second consecutive run reporting the same stalled digest, and the backlog is now three days deep). Items across the four digests in the window that belong to developments-log: Anthropic's $35bn Lambda cloud agreement with Nvidia holding the Texas lease; ChatGPT Mil on GenAI.mil as the Defense Department's second approved generative-AI tool; the European Commission's DSA designation of ChatGPT as a very large online search engine alongside Reddit and Roblox; FSB Chair Andrew Bailey's letter to the G20; Trump's "let Data Reign" post and the data-center counter-campaign; Ethan Perez's and Jack Clark's remarks on the Hugging Face investigation; and Ernst & Young's $100m "human skills" bonus programme.
  • entities/adam-thierer (score 3) — fourth consecutive deferral, confirmed structural. Reported as structural on 2026-08-31; unchanged. Named on Federal Trade Commission (FTC) and Radical Optionality: Governing Transformative AI Under Uncertainty. It will keep losing to live-thread candidates unless a third source appears or it is created deliberately. Not re-argued here.
  • Slice-5 thinness, the corrected diagnostic. The mechanical test (in-degree ≥6 with sources_count ≤ 2 or no ## Relationships) returns 16 of 140 pages in entities/ H–N — a workable number, unlike slices 3 and 4, because every page in the slice already has a Relationships section. Reading the top of the list confirms most are correctly short person pages. The genuine remainder, deferred: Helen Toner (in-degree 6, 530 words, confidence: high on sources_count: 1 — a schema violation, and zero inline citations anywhere on the page, last_updated: 2026-06-06); International Monetary Fund (IMF) (in-degree 9, 398 words, sources_count: 1, and still carrying a "Stub created 2026-05-11" line under a ## Sources header); Henry Farrell (in-degree 13, 853 words, sources_count: 2, last_updated: 2026-06-06). Toner is the strongest of the three and needs a sourcing pass rather than a link pass — its defect is that nothing on it is cited, which this run's remaining budget could not fix honestly.
  • Slice-4 carryover, second deferral: American Civil Liberties Union (ACLU) (in-degree 24, 439 words, sources_count: 2) and Access Now (in-degree 20, 446 words, sources_count: 2).
  • Three slice-3 pages, third deferral (scores 3, 2, 2): Standards-Industrial Complex, Three Theories of Victory (US / EU / China AI Governance), Transaction Costs (Coase) and the Agentic Economy, all sources_count: 1 at in-degree 6–8. Structural by the step-4 rule at three consecutive deferrals; stated once here rather than cycled again.
  • The AI Evaluator Forum and its AEF-1 evaluator-disclosure standard (score 4) — new this run. Surfaced while verifying the Transluce report, which publishes an AEF-1-conformant disclosure and a template of its lab agreements. A published standard for third-party evaluator independence, access and transparency belongs in standards/, and the vault has nothing on it. Recorded in the Transluce ingest task so it is not lost.
  • ExploitBench flag-level disclosure (score 3) — new this run. Whether developers publish the flag-level results behind their single-percentage ExploitBench scores is now an open question on ExploitBench; if they do, the vault's comparison tables can be made comparable.
  • Twenty-one source-robustness-check candidates across slices 11–13 — ninth consecutive report, structural. Unchanged, led by California SB 53 — Transparency in Frontier AI Act (in-degree 254) and TAKE IT DOWN Act — Source Summary (97). Out of this skill's lane.
  • models/deepseek-v3 — eleventh run, structural. In-degree 49 on 608 words, routed to model-robustness-check on 2026-08-22, still unaddressed.
  • The 28 date-stamped headers in lint-report.md (score 1) — sixteenth cycle, structural. Unchanged. The check does not distinguish a litigation procedural history from a news-feed accretion; fixing the finding means fixing bin/lint-scan.py.
  • Carried unchanged. Marvell (4 mentions, no company page, carried since 2026-08-21); California AB 1856 (text still not located); the Texas OAG / Meta Webster–Mahoney email exchange of 2026-08-25; the draft executive order creating a self-regulatory organization for frontier developers, still unactionable because no public text exists.
  • Examined and declined, not deferred. "Hut 8" returns zero mentions across all content folders despite appearing in the 2026-08-31 Anthropic–Lambda item. A company the graph has never referenced is not a gap in it; it is an item for the nightly fold to introduce. Same disposition as DuckLabs on 2026-08-30.

Run integrity

  • No broken outbound links introduced. Every wikilink on the three new pages, the two expanded pages, the twelve wired pages, the two INGEST- tasks, the two verification records and the needs-review note was resolved after writing against the same target set bin/lint-scan.py uses: zero unresolved on content pages. Three unresolved appear in the check output and all three are accounted for — two are pre-existing on The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity? and were not introduced by this run's single additive edit to it, and one is a deliberate forward reference to sources/exploitbench-lee-brumley-2026 inside the INGEST- task, which lives in _meta/ and is not linted as content.
  • Broken-link count fell 213 → 212 distinct. The referenced-2+ lane was empty before the run and is empty after.
  • Style scan clean, first pass. All five new and expanded pages checked programmatically against the banned-header, hyperbole, self-reference, standing-Predictions and date-led-header lists: zero hits on each. Wiki-wide after the run: banned_headers=0, self_refs=0, predictions=0, header_hype=0, date_headers=28.
  • Revision fidelity. Twelve of the fourteen edits to existing pages were single wikilink wraps around text already present. Of the two substantive revisions: John Thune was purely additive, no sentence removed. Jared Kaplan preserved every fact and every link in the original — including the TechCrunch interview of June 6, 2025, which was kept without a URL because the original carried none and inventing one would be worse — with one deliberate removal, recorded here rather than made silently. The original read that Kaplan "is among the more influential technical figures in AI, and is less publicly visible than Dario Amodei while remaining foundational to Anthropic's research strategy." That is an unsourced editorial characterization of a living person, banned by house-style §1 and §2, and carried no citation. It was deleted rather than relocated. The Dario Amodei link it contained survives elsewhere on the page.
  • Confidence set honestly, including one correction downward. The three new pages are all confidence: medium; none met the three-independent-source bar. Jared Kaplan was lowered from high to medium: its high rating rested on sources_count: 2, below the schema's own bar for that rating, and gap-identifier does not set high on its own authority. The page now carries 8 sources and may warrant high on a curated review; this run did not grant it.
  • A single-source governance claim marked as such rather than absorbed. Observer's report that Kaplan has been Anthropic's responsible scaling officer since 2024 is attributed inline, and the page states that Anthropic has published no corresponding confirmation. The RSP text the vault holds defines the role without naming a holder.
  • Cap respected. Eight gaps actioned against a 6–10 cap; 27 candidates found. Six successful web pulls and four searches; two pulls exceeded the sandbox buffer and were handled by structured extraction or abandoned; no crawls.
  • Dashboard rebuilt — 2,096 article pages written to Wiki/_meta/site.nosync/.

One-line summary

Three pages created and two expanded live, two foundational sources queued with full verification, and ten source-fidelity divergences recorded — six of them against Transluce, which the nightly fold wrote from secondary accounts on 2026-08-31 and which misstates the study's scope, its release status and its lab-access terms; the curator's attention is most needed on the Transluce ingest and on the 2026-08-30 22:10 digest, unprocessed for a third day.

Categories: Briefings | briefing | gap-scan