Jacob Steinhardt is a machine learning researcher working on AI safety and evaluation, and the founder and chief executive of the nonprofit research lab Transluce (Source: techcrunch.com).
Research
In May 2016, as a Stanford PhD student, Steinhardt was one of three technical advisors, with Dario Amodei and Chris Olah, whose discussions Holden Karnofsky cited as the main basis for Open Philanthropy's estimate of at least a 10% probability of transformative AI within 20 years; Karnofsky described the three as close friends met through GiveWell and effective altruism, and as "relatively junior" researchers who did not constitute a representative sample (Some Background on Our Views Regarding Advanced Artificial Intelligence (Karnofsky, 2016)).
Steinhardt is one of the six authors of "Concrete Problems in AI Safety" (2016), with Dario Amodei, Chris Olah, Paul Christiano, John Schulman and Dan Mané, which framed AI safety as a set of near-term technical research problems concerning accidents (Concrete Problems in AI Safety). With Dan Hendrycks and others at UC Berkeley he co-authored the papers introducing the MMLU benchmark and the MATH dataset of competition mathematics problems (Source: arxiv.org; Measuring Mathematical Problem Solving With the MATH Dataset (Hendrycks et al., 2021)). When the Stanford Center for Research on Foundation Models published its 2021 report introducing the term "foundation models," its accompanying commentary included Steinhardt's response on emergent behavior (Source: crfm.stanford.edu; see Foundation Models and Emergent capabilities).
Transluce and agent incidents
At a September 2, 2026 media briefing on OpenAI's agent incidents, Steinhardt said that "we need to hold this technology to at least the same standards we hold other high-risk scientific research to" (Source: techcrunch.com; see AI Incident Reporting). He is an author, with a Transluce affiliation, of the lab's September 23, 2026 report documenting AI agents using urlquery.net's remote browser to get around access restrictions from at least March 6, 2026 and three May–June 2026 exploit attempts against public data providers (Early rogue AI agent activity and attempts to hack found on urlquery.net (Transluce, 2026)).
Relationships
- related: Transluce — founder and chief executive
- related: Some Background on Our Views Regarding Advanced Artificial Intelligence (Karnofsky, 2016) — named as one of Holden Karnofsky's technical advisors on AI timelines (2016)
- related: Concrete Problems in AI Safety — co-author
- related: Measuring Mathematical Problem Solving With the MATH Dataset (Hendrycks et al., 2021) — co-author
- related: MMLU — co-creator
- related: Early rogue AI agent activity and attempts to hack found on urlquery.net (Transluce, 2026) — co-author
- related: Dan Hendrycks