Open Problems in Technical AI Governance is a 2024 survey by Reuel, Bucknall, Casper, Fist, and more than 50 co-authors affiliated with Stanford, the Centre for the Governance of AI, MIT, and other institutions. The 94-page paper catalogs open technical problems in AI governance — technical analysis and tools that can support effective governance of AI — across areas including model evaluation, data governance, compute governance, hardware security, and AI safety.
The paper was first posted to arXiv (2407.14981) on 20 July 2024 and was revised on 16 April 2025; it was published in the Transactions on Machine Learning Research in 2025 under a CC BY 4.0 license (Source: https://arxiv.org/abs/2407.14981). Anka Reuel (Stanford) and Ben Bucknall contributed equally and share the first-author position; the listed author set comprises 33 named contributors (Source: https://arxiv.org/abs/2407.14981). The paper is published by the Centre for the Governance of AI (GovAI), which describes it as intended "as a resource for technical researchers or research funders looking to contribute to AI governance" (Source: https://www.governance.ai/research-paper/open-problems-in-technical-ai-governance).
Authorship
The named authors are Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Jeffrey Ladish, David Bau, Paul Bricman, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Sanmi Koyejo, Mykel J. Kochenderfer, and Robert Trager (Source: https://arxiv.org/abs/2407.14981). Co-author Lisa Soder published an executive summary of the paper through the European technology-policy think tank Interface (formerly Stiftung Neue Verantwortung) in December 2024 (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
Summary
The paper defines technical AI governance as "technical analysis and tools for supporting the effective governance of AI" and argues it can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance (Source: https://arxiv.org/abs/2407.14981). The authors frame the field as a response to two gaps that they argue constrain current governance: a gap in technical information needed to identify where interventions are warranted and to assess policy options, and a gap in the technical tools needed to implement policy proposals (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
The authors position technical AI governance as one component of a broader governance portfolio rather than a standalone solution. Soder's summary states that the field "should be seen in service of broader sociotechnical and political solutions" and that "a purely 'techno-solutionist' approach to AI governance and policy is unlikely to succeed" (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
Taxonomy
The paper organizes technical AI governance along two dimensions: technical targets across the AI value chain (from inputs to deployment) and governance capacities that can be applied to those targets (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
The four targets are:
- Data — the datasets used for training AI models.
- Compute — the computational resources required for AI development (Compute governance).
- Algorithms and Models — the software and mathematical frameworks underpinning AI systems.
- Deployment — the real-world application and integration of AI systems.
(Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)
The six governance capacities are:
- Assessment — evaluating AI systems for safety, fairness, robustness, and effectiveness.
- Access — controlling and facilitating appropriate access to AI systems and data, balancing openness against security and privacy.
- Verification — ensuring integrity, compliance, and accountability through mechanisms such as audits and certifications.
- Security — protecting AI system components from unauthorized access, manipulation, and tampering.
- Operationalization — translating ethical principles, legal requirements, and policy objectives into concrete technical strategies, procedures, or standards.
- Ecosystem Monitoring — observing and analyzing trends in AI development, deployment, and impact to inform forward-looking governance.
(Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)
Representative open problems
For each capacity, the paper enumerates open research questions across the relevant targets. Examples drawn from Soder's executive summary include:
- Assessment. What license and metadata reporting requirements could assist responsible data practices, and what infrastructure would let researchers audit large datasets? Which hardware properties or chip specifications are most indicative of suitability for AI training or inference? How can the thoroughness of model evaluations be measured, and how can data contamination be accounted for? How can downstream impact evaluations be scaled across languages and modalities (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
- Access. How can data access preserve privacy while enabling meaningful auditing? How can public compute resources be allocated fairly between users? What auditing methodologies are possible along the continuum between black-box and white-box model access (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
- Verification. How can it be verified that a model was (or was not) trained on a given dataset? How can metadata watermarking be applied to AI-generated content, and how should verification information from model registries be presented to users (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
- Security. How can AI systems be made robust to data-extraction attacks? How can hardware-enabled governance methods be implemented at scale, and what infrastructure-level measures can protect model weights from theft by an adversary (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
- Operationalization. Which system properties, if any, are the most reliable indicators of risk and thus candidates for serving as regulatory targets? What intervention and correction options exist if flaws are identified post-deployment (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
- Ecosystem Monitoring. How can trends observed in current systems be extrapolated to make predictions about future systems? How could developments in AI-specific hardware affect the governability of compute, and what information is needed to assess the environmental impact of AI development and deployment (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance)?
Key claims
The paper's stated takeaways include:
- Evaluations of systems and their downstream impacts feature in many proposed governance regimes, but the authors argue that current evaluations lack robustness, reliability, and validity, especially for foundation models (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
- Hardware mechanisms could potentially enable privacy-preserving access to datasets and models, verification of compute use, or attestation of audit and evaluation results, but the authors state that the use of such mechanisms for these purposes is "largely unproven" (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
- The development of AI research infrastructure — for analyzing large training datasets or providing privacy-preserving access to models for evaluation and auditing — could support scientific understanding of AI systems and external oversight of developers (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
- Ecosystem-monitoring research that collects and analyzes data on AI trends has, the authors argue, already proven useful for helping policymakers keep policy forward-looking (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
The recommendations accompanying these takeaways call for allocating funding to technical AI governance research through open calls and funding bodies; closer collaboration between policymakers and technical experts; in-house technical research by bodies such as AI Safety Institutes and the EU AI Office, beyond their evaluation work; and attention to technical AI governance at venues including future AI summits, the G7, the UN AI advisory body, and the International Scientific Report on the Safety of Advanced AI (Source: https://www.interface-eu.org/publications/open-problems-in-technical-ai-governance).
Key topics
The survey catalogs open problems across several domains:
- Model evaluation and benchmarking limitations
- Data governance, including provenance, privacy, and consent
- Compute governance, including monitoring, allocation, and access control
- Hardware security and supply chain
- AI safety evaluation methods
- Transparency and interpretability for governance
- International coordination mechanisms
Among these, the paper provides a detailed technical analysis of compute as a governance lever (Compute governance). Its treatment of AI safety evaluation methods relates to the cyber threshold framework described in Managing Cyber Risks, and its catalog of technical problems can be mapped onto the Govern, Map, Measure, and Manage functions of the NIST AI RMF 1.0. The technical problems it identifies also underpin the regulatory options compared in US Regulatory Approaches.
Provenance
Authored by Reuel, Bucknall, Casper, Fist, et al. (Stanford, Centre for the Governance of AI, MIT, and others), 2024. PDF converted to markdown on 2026-04-13. The canonical version is arXiv:2407.14981 (v1 submitted 20 July 2024; v2 revised 16 April 2025), published in Transactions on Machine Learning Research, 2025, under CC BY 4.0 (Source: https://arxiv.org/abs/2407.14981). Published by the Centre for the Governance of AI (Source: https://www.governance.ai/research-paper/open-problems-in-technical-ai-governance).