"NIST Just Told Us What's Actually Broken in AI Governance" is an essay by Andrew Clearwater, published March 10, 2026 on his Substack (Source: https://andrewclearwater.substack.com/p/nist-just-told-us-whats-actually). It is a practitioner-oriented walkthrough of NIST AI 800-4, written for enterprise AI-governance audiences. Clearwater characterizes the report as "one of the most important AI governance documents published in 2026 so far, and it's going to be underread because it identifies problems rather than prescribing solutions. That's exactly why it matters."
Summary of argument
Clearwater organizes his reading of the report around three high-level claims.
First, he argues that pre-deployment testing is necessary but not sufficient. AI evaluations done before release are, in his account, predominantly conducted in controlled environments that cannot account for real-world dynamics; models are non-deterministic and behave differently when they detect being evaluated. He writes that practitioners relying primarily on pre-deployment evaluation results to justify risk decisions now have a "big asterisk" next to their methodology.
Second, he argues that the center of gravity in AI governance is shifting from pre-deployment to post-deployment. In his framing, the governance function is becoming a continuous process running for a system's life rather than a gate before launch, and governance teams structured as pre-launch review boards are, in his words, building for the wrong era.
Third, he argues that the report's six monitoring categories supply a shared language the field has lacked. He singles out the "Does the system promote human flourishing?" anchor in category six as a normative question embedded in a federal technical report.
Key concepts surfaced
Clearwater highlights six concepts drawn from NIST AI 800-4:
- Goodhart's Law and the Streetlight Effect — which he describes as formalized as monitoring barriers in a federal report for the first time.
- Privacy-monitoring paradox (the "privacy vs. granularity trade-off") — which he frames as an unsolved structural problem.
- Monitorability tax — the performance and cost penalty for making agents monitorable, which he expects to hit agents hardest.
- Shadow AI — employee use of personal AI accounts and devices outside sanctioned tools.
- Disagreement about what an "AI incident" is — no shared definition, no centralized reporting infrastructure, and over-indexing on newsworthy incidents.
- Information sharing as the limiting factor — developers do not know how their models are used downstream, and deployers lack upstream visibility.
Practitioner recommendations
The essay closes with steps Clearwater directs at governance practitioners: audit monitoring coverage against all six categories, noting that most organizations are strong on functionality and security but weak on human factors and large-scale impacts; build post-deployment monitoring into the AI lifecycle from the start; begin tracking human-AI interaction patterns, which he calls the least mature area and argues carries the highest stakes; and develop an incident taxonomy without waiting for field-level consensus, on the basis that NIST has flagged that shared definitions do not yet exist.
Clearwater also presents the essay as translating the federal report into actionable implications for governance practitioners and as surfacing its practitioner-level concepts before they would otherwise be widely read. The piece is positioned as part of a recurring effort by Clearwater to interpret NIST guidance for enterprise audiences.
Relationships
- analyzes: NIST AI 800-4: Challenges to the Monitoring of Deployed AI Systems
- introduces: Post-Deployment AI System Monitoring (this concept), Monitorability Tax, Shadow AI (in practitioner-readable form)
- authored-by: Andrew Clearwater
- related: System Card Due Diligence, Alignment Risk Update (parallel framework)