A foundational academic essay by Deirdre K. Mulligan (UC Berkeley iSchool), Nik Marda (UC Berkeley), and Victor Zhenyi Wang (UC Berkeley), published March 16, 2026 by the Knight First Amendment Institute (Knight Columbia) / Knight Columbia. The essay argues that current AI risk governance lacks a conceptual framework for reasoning about where to intervene, and that this absence pushes existing frameworks toward component-by-component, model-centric mitigations. It proposes a sociotechnical-systems orientation, distinguishes "hazards" (probability of future harm at the component level) from "harms" (realized negative impact), argues for a preference for harm-reducing interventions over hazard-reducing ones, and applies the framework to image-based sexual abuse (AI-generated non-consensual intimate imagery, AI-NCII) through applied case studies.
Central thesis
The authors argue that AI risk governance is stymied by the absence of a conceptual framework to reason about intervention sites — data, models, applications, organizational processes, and policies. Without that framework, the essay contends, existing frameworks such as the EU AI Act, US OMB M-24-10 (Young 2024), the UK AISI Research Agenda, the NIST AI RMF, and voluntary AI commitments default to component-by-component, model-centric, technocratic mitigations that may reduce hazards (probability of future harm) without composing to reduce harms (realized negative impact).
The essay proposes two analytic shifts:
- A sociotechnical-systems orientation, in which risk assessment accounts for the technical and organizational context (Dobbe 2022), and harms emerge from interactions among technical, social, political, and economic components rather than from a system output alone.
- A preference for harm-reducing interventions over hazard-reducing ones, paired with disentangling ownership and control of AI models and systems from participation in risk management, to make room for entities with relevant expertise, operational capacity, and independence.
The four critiques of current AI governance
A. Component-by-component hazard reduction may not reduce harms. Most frameworks identify objects to protect (rights, safety, democracy, environment) but allow entities to direct mitigation as they see fit. The resulting component analysis lacks coordination, with entities focusing on the technical artifact they develop or deploy. The authors characterize the dominant "if-then" logic as positing the model capability as the hazard and then deriving mitigations centered on the model; their claim is that "Good governance ought to do the reverse" — start from the harm, then trace the assemblage that produced it. They illustrate with two analogies. In the Hurricane Katrina case, emphasis on the levee as hazard led to $14.5B in flood-control investment, but the 1,400 deaths were coproduced by the hurricane, levee failure, hollowed-out public services, and crumbling infrastructure; component-focused mitigation produces a stilted view of what to fix. In the 2025 Central Texas floods, Kerr County vulnerability was shaped by social and political conditions, not the flash flood alone.
B. Risk management in the hands of developers and deployers limits expertise and undermines legitimacy. The essay notes that the EU AI Act, U.S. OMB AI guidance, and voluntary AI commitments largely leave regulated entities in charge of risk management. It engages with Abbott & Snidal (2009) on Regulatory Standard-Setting (RSS), arguing that firms lack independence, normative expertise, and representativeness beyond economic stakeholders; with Bamberger (2006) on "cognitive decisionmaking pathologies," whereby firm processes filter out information about risk and change that regulation seeks to identify; and with Solow-Niederman (2020) on an era of "private governance" that the authors argue will prevent public values from informing AI research, development, and deployment.
C. Overly narrow and technocratic risk management tools and practices. The authors argue that CAISI (formerly US AISI), AISI (UK), and sister institutes have drifted toward model-centric, technocratic evaluations despite the NIST AI RMF's framing of risk management as "coordinated activities to direct and control an organization." They note that public-sector AI safety institutes have leaned heavily on private-sector approaches — pre-release evaluations (HarmBench, Cybench, LAB-bench), red-teaming, and weight, training-data, or training-code modifications. The published pre-release AISI+CAISI joint evaluation of OpenAI o1 and Anthropic Claude Sonnet 3.5 is cited as paradigmatic, framed around "capabilities and potential impacts" rather than harm-orientation and reliant on industry-standard benchmarks rather than independent socio-technical assessment. Combined with private-sector deference, the authors argue this produces "regulatory managerialism" (Cohen & Waldman 2023) — importing private-sector practices and underlying ideologies into regulated activities.
D. Constrained view on potential mitigation sites. The essay argues that frameworks over-emphasize model assessments and mitigation strategies. The EU AI Act, Biden EO 14110, and voluntary commitments concentrate on model providers and high-risk-AI-system deployers as risk-bearing actors, which the authors say occludes the full spectrum of hazards and their relations to particular harms.
The applied case: AI-NCII (image-based sexual abuse exacerbated by AI)
Part III of the essay applies the framework to image-based sexual abuse, also termed AI-generated non-consensual intimate imagery (AI-NCII). The authors contrast a model-centric intervention — fine-tuning out the capability (cited: Gandikota et al. 2023, Thiel 2023), which they describe as useful but easily circumvented — with sociotechnical interventions: platform-side accountability, payment-processor cooperation, cross-jurisdictional regulatory coordination, and survivor support infrastructure.
This case connects to It's Too Soon To Tell If the TAKE IT DOWN ACT Is Working (Cuevas, Tech Policy Press, May 13 2026) (Cuevas's TIDA-1-year empirical status), which finds that despite federal criminalization (TAKE IT DOWN Act, signed by Trump May 19 2025), supply and demand for AI-NCII grew across 4chan, Website A, and Website B in 2025. Cuevas's finding that platform-side accountability is the operative deterrence lever aligns with the sociotechnical-system claim advanced by Mulligan, Marda, and Wang.
The four policy recommendations
- Develop a sociotechnical-system map that identifies technical and organizational system components related to the harm under exploration.
- Deployers should assess and mitigate risks of AI use cases, not systemic risks — situating risk at the deployment level rather than the model level.
- Reduce reliance on developers and deployers to independently engage in risk mitigation; incentivize entities to enlist external stakeholders with risk-relevant expertise, and include external stakeholders in strategic decisions and, where relevant, directly in risk-mitigation activities.
- Governments and companies should invest in infrastructure and research to support sociotechnical evaluations and the richer technical and non-technical risk-mitigation techniques required to reduce harms.
Engagement with existing positions
The essay engages directly with model-centric framing in Responsible Scaling Policy (RSP), Anthropic's Responsible Scaling Policy (Version 2.2), OpenAI Preparedness Framework V.2, and Safety Cases (Frontier AI) (AISI inability-arguments per Goemans et al. 2024, Clymer et al. 2025), which Mulligan, Marda, and Wang treat as paradigmatically model-centric and if-then-framed, though they would credit these as useful within their proper scope. The essay also cites Mulligan's prior work on handoffs as an analytic approach compatible with the sociotechnical perspective, relevant to Sociotechnical AI Risk Governance.
The essay engages substantively with International AI Safety Institute Network (INSAI / AISIN), NIST CAISI (Center for AI Standards and Innovation), and NIST AI Risk Management Framework 1.0, arguing that these institutions have drifted from the NIST AI RMF's organizational risk-management framing toward private-sector-imported technical evaluations. The AISI "cyber misuse" framing discussed at AI and Cybersecurity is, in the authors' view, an instance of the technocratic, model-centric pattern they critique.
As a position piece, the essay advances the sociotechnical-AI-risk-governance argument and the harm-versus-hazard distinction as a methodological primitive, connecting to AI risk management. It sits in tension with Anthropic's Responsible Scaling Policy (Version 2.2) and OpenAI Preparedness Framework V.2, which the authors read as model-centric and if-then-framed.
Relationships
- supports: Sociotechnical AI Risk Governance
- engages-with: Anthropic's Responsible Scaling Policy (Version 2.2), OpenAI Preparedness Framework V.2, Safety Cases (Frontier AI) (treats them as paradigmatically model-centric)
- engages-with: NIST CAISI (Center for AI Standards and Innovation), International AI Safety Institute Network (INSAI / AISIN) (the technical-drift critique)
- applies-to: Synthetic Media / Deepfakes, TAKE IT DOWN Act (AI-NCII case study)
- related: NIST AI Risk Management Framework 1.0, Regulatory Managerialism (Cohen-Waldman framing), Safetywashing, Sociotechnical Systems, Deirdre K. Mulligan, Nik Marda, Victor Zhenyi Wang, Knight First Amendment Institute (Knight Columbia)
- part-of cluster: Knight Columbia 2026 AI-in-democratic-society symposium
Sources
Raw Sources/A Conceptual Model to Guide AI Risk Governance Strategies.md- Published at Knight Columbia: knightcolumbia.org