AI Policy Wiki
Dashboard
⚲
AI Policy Wiki › Concepts & Frameworks

Sociotechnical AI Risk Governance

medium confidence · updated 2026-07-19

Framework by Deirdre Mulligan, Nik Marda, and Victor Zhenyi Wang (Knight Columbia, March 2026) critiquing model-centric AI risk governance and proposing a sociotechnical-systems orientation. Names the harm/hazard distinction as a methodological primitive (harms = realized negative impacts; hazards = probabilities of future harm) and the 'handoff lens' as an analytic approach.

Contents

Sociotechnical AI risk governance is a framework proposed by Deirdre K. Mulligan, Nik Marda, and Victor Zhenyi Wang in A Conceptual Model to Guide AI Risk Governance Strategies (A Conceptual Model to Guide AI Risk Governance Strategies (Mulligan + Marda + Wang, Knight Columbia, March 16 2026), Knight Columbia, March 2026). It critiques what the authors characterize as model-centric AI risk governance and argues instead for a sociotechnical-systems orientation in which risk is assessed across the technical and organizational context that produces harm. The framework rests on two methodological primitives: a distinction between harm and hazard, and a sociotechnical-systems orientation.

Core distinctions

The framework defines harms as realized negative impacts and hazards as probabilities of future harm. Mulligan, Marda, and Wang argue that good governance should begin from realized harm and trace back to the assemblage that produced it, rather than positing a model capability as a hazard and deriving model-centered mitigations from it.

The second primitive is a sociotechnical-systems orientation: risk assessment must account for technical and organizational context (citing Dobbe 2022), because harms emerge from interactions between technical, social, political, and economic components rather than from a system output alone. The components named include the model, training data, organizational processes, professional training, deployment configuration, user interaction patterns, the regulatory environment, and professional norms. The authors treat these as nodes in an assemblage and locate the hazards that lead to harm in the interactions between them.

Critique of existing risk-governance frameworks

The authors' central critique is that the EU AI Act, US OMB M-24-10, the UK AISI Research Agenda, NIST AI RMF, and voluntary AI commitments default to component-by-component, model-centric, technocratic mitigations that may reduce hazards without composing to reduce harms. They identify four failures.

First, component-by-component hazard reduction may not reduce harms. The authors describe an "if-then" logic that posits a model capability as a hazard and derives mitigations centered on the model, and argue that "good governance ought to do the reverse" by starting from the harm and tracing the assemblage that produced it. They use Hurricane Katrina as an analogy: an emphasis on the levee as the hazard produced $14.5B in flood-control investment, but the 1,400 deaths were coproduced by the hurricane, levee failure, hollowed-out public services, and crumbling infrastructure.

Second, leaving risk management in developers' and deployers' hands limits expertise and undermines legitimacy. The authors note that the EU AI Act, U.S. OMB AI guidance, and voluntary AI commitments largely leave regulated entities in charge. They draw on Abbott & Snidal (2009) on regulatory standard-setting (firms lack independence, normative expertise, and representativeness beyond economic stakeholders), Bamberger (2006) (firm processes filter out information that regulation seeks to identify), and Solow-Niederman (2020) (an era of "private governance" prevents public values from informing AI research).

Third, the tools and practices are overly narrow and technocratic. The authors argue that CAISI (formerly US AISI), AISI (UK), and sister institutes have drifted toward model-centric, technocratic evaluations despite NIST AI RMF's framing of risk management as "coordinated activities to direct and control an organization." They point to reliance on industry-standard benchmarks (HarmBench, Cybench, LAB-bench) for pre-release evaluations, and to red-teaming being treated methodologically as a safeguard posited to directly mitigate risk when, in their account, red-teaming is the discovery of potential weaknesses rather than a guarantee of their absence. They invoke "regulatory managerialism" (Cohen & Waldman 2023), the importing of private-sector practices and underlying ideologies into regulated activities.

Fourth, the frameworks take a constrained view of potential mitigation sites. The authors argue that over-emphasis on model assessments and mitigation occludes the full spectrum of hazards and their relations to particular harms.

The two analytic shifts

The framework proposes two shifts. The first is the sociotechnical-systems orientation described above, under which risk assessment accounts for the technical and organizational context that coproduces harm and locates hazards in the interactions among components.

The second is a preference for harm-reducing over hazard-reducing interventions. Because hazard reduction at the component level may not compose to harm reduction, the authors argue that policy frameworks should prefer interventions (and collections of interventions) that target realized harms, disentangle ownership and control of AI models from participation in risk management, and enlist external stakeholders with risk-relevant expertise, operational capacity, and independence in risk-mitigation strategy and execution.

The handoff lens

Mulligan's prior work on handoffs is named as the analytic approach compatible with the sociotechnical perspective. A handoff is the moment in a sociotechnical system at which responsibility for a function is transferred between components (human to AI, AI to human, AI to AI, organization to organization). The authors treat handoffs as the site of much of the risk in sociotechnical systems and as an analytic primitive, asking "where are the handoffs?" rather than "what are the model's risks?"

The handoff lens connects to Levels of Autonomy for AI Agents (Feng-McDonald-Zhang framework) (Feng-McDonald-Zhang), in which each level's user-role is structured around handoff patterns; to Principal-Agent Problem Applied to AI; and to Meaningful Human Review, where Colorado SB 26-189 (2026 — replaces 2024 Colorado AI Act)'s §6-1-1701(15) definition is treated as a handoff design.

Policy recommendations

The framework advances four policy recommendations. First, develop a sociotechnical-system map that identifies the technical and organizational system components related to the harm. Second, have deployers assess and mitigate the risks of AI use cases rather than systemic risks, situating risk at the deployment level rather than the model level. Third, reduce reliance on developers and deployers to independently engage in risk mitigation, incentivizing entities to enlist external stakeholders with risk-relevant expertise and including those stakeholders in strategic decisions and, where relevant, directly in risk-mitigation activities. Fourth, governments and companies need to invest in infrastructure and research to support sociotechnical evaluations and the richer technical and non-technical risk-mitigation techniques required to reduce harms.

Applied case: AI-NCII

Part III of the paper applies the framework to image-based sexual abuse exacerbated by AI (AI-NCII). The authors contrast a model-centric intervention—fine-tuning out the capability, which they describe as useful but easily circumvented (Wei et al. 2024)—with sociotechnical interventions spanning platform-side accountability, payment-processor cooperation, cross-jurisdictional regulatory coordination, survivor support infrastructure, criminal prosecution, and civil liability.

The case study connects to It's Too Soon To Tell If the TAKE IT DOWN ACT Is Working (Cuevas, Tech Policy Press, May 13 2026), in which Cuevas's one-year empirical status of the TAKE IT DOWN Act (TIDA) finds that, despite federal criminalization, supply and demand for AI-NCII grew across 4chan, Website A, and Website B in 2025. Cuevas identifies platform-side accountability as the operative deterrence lever, which the authors read as supporting the sociotechnical-system claim.

Relation to other governance frameworks

Mulligan, Marda, and Wang engage several existing frameworks tracked elsewhere. They treat Responsible Scaling Policy (RSP), Anthropic's Responsible Scaling Policy (Version 2.2), and OpenAI Preparedness Framework V.2 as paradigmatically model-centric and if-then-framed—useful within their proper scope but, in the authors' view, insufficient as the dominant governance frame. They critique International AI Safety Institute Network (INSAI / AISIN) and NIST CAISI (Center for AI Standards and Innovation) for drifting from NIST AI RMF's organizational risk-management framing toward private-sector-imported technical evaluations, and treat NIST AI Risk Management Framework 1.0 as a framework whose breadth has been under-utilized by its sister institutes. They also engage Safety Cases (Frontier AI) and inability arguments (Goemans et al. 2024; Clymer et al. 2025), positioning these as if-then-flavored.

Position in the sociotechnical literature

The sociotechnical orientation the paper builds on predates it in AI risk work. NIST's AI Risk Management Framework 1.0 (January 2023) characterizes AI systems as inherently socio-technical, influenced by societal dynamics and human behavior as well as technical components (Source: nvlpubs.nist.gov) — the framing Mulligan, Marda, and Wang argue the AISI-network institutes have under-utilized. In evaluation research, Weidinger et al. (Google DeepMind, October 2023) proposed a three-layer sociotechnical framework for safety evaluation of generative AI in which model-capability evaluation is complemented by evaluation of human–system interaction and of systemic impact, on the ground that capability testing alone leaves a gap when risks emerge in context (Source: arxiv.org). The Mulligan–Marda–Wang framework is distinct from this evaluation-layer work in extending the sociotechnical claim from how systems are evaluated to where governance should intervene — preferring harm-tracing and deployment-level intervention over model-level hazard reduction — but shares its premise that model-only analysis under-captures risk. The paper's own citation for the sociotechnical-systems orientation is Dobbe (2022), noted above.

Relationships

Sources

Confidence note: Medium — the framework itself rests on a single canonical source (a March 2026 symposium paper); the supplementary sources establish the surrounding sociotechnical literature, not independent corroboration of the framework's claims. Previously misrated high on sources_count 1.

Categories: Concepts & Frameworks | global | framework | regulation | ethics | safety
What links here (27)