Holden Karnofsky is a co-founder of Open Philanthropy (renamed Coefficient Giving in November 2025) and led its work on AI worldview and strategy (Open Philanthropy). He introduced the term "transformative AI" in a May 2016 Open Philanthropy post that framed the organization's decision to prioritize potential risks from advanced AI (Some Background on Our Views Regarding Advanced Artificial Intelligence (Karnofsky, 2016)), and he later worked on AI safety in roles associated with the Carnegie Endowment for International Peace and Anthropic (RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts (Wijk et al., 2024)).
Roles
At Open Philanthropy, Karnofsky is described as a co-founder and longtime lead of the AI worldview and strategy work (Open Philanthropy); a compilation of stated P(doom) estimates lists him as executive director of Open Philanthropy at the time of his estimate (P(doom)). The Open Philanthropy page states that he moved in 2024 to work on AI safety directly in roles adjacent to Anthropic. The RE-Bench paper, dated 20 November 2024, lists him among its authors with an Anthropic affiliation and notes that his work was done while at the Carnegie Endowment for International Peace (RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts (Wijk et al., 2024)). He is named as a major contributor to Claude's Constitution, alongside Chris Olah and Jared Kaplan, with Amanda Askell as primary author (Claude's Constitution), and he is credited with feedback on Anthropic's 2026 publication on measuring the pace of AI development (Measurements for understanding the pace of AI development inside frontier labs (Anthropic Institute, September 2026)).
Writing on transformative AI
In "Some Background on Our Views Regarding Advanced Artificial Intelligence" (May 6, 2016), Karnofsky defined transformative AI as "AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution," with a more detailed three-part definition covering AI able to do the work of human scientists unaided in developing a transformative technology, AI able to perform tasks accounting for the majority of full-time jobs or over 50% of world wages at comparable cost, and AI-centric technologies such as surveillance or autonomous weapons that themselves drive such a transition. He put the probability of transformative AI within 20 years at "at least 10%," held with moderate robustness, and at least 1% with high robustness, basing the estimate mainly on discussions with technical advisors including Dario Amodei, Chris Olah and Jacob Steinhardt (Some Background on Our Views Regarding Advanced Artificial Intelligence (Karnofsky, 2016)). The definitions became the basis of the concept of transformative AI used across later governance and economics writing (Transformative AI (TAI)). He later framed the stakes of transformative AI in the "Most Important Century" blog series (Open Philanthropy).
Positions and statements
A compilation of public P(doom) estimates, drawing on Wikipedia's collation of interviews and posts, lists Karnofsky at 50% (P(doom)).
Relationships
- related: Open Philanthropy — co-founder and longtime leader
- supports: Transformative AI (TAI) — originated the term (Some Background on Our Views Regarding Advanced Artificial Intelligence (Karnofsky, 2016))
- related: AGI Timelines — 2016 estimate of at least 10% probability of transformative AI within 20 years
- related: Anthropic — later affiliation; contributor to Claude's Constitution
- related: Dario Amodei, Chris Olah, Jacob Steinhardt — technical advisors named in the 2016 post
- related: RE-Bench — co-author of the RE-Bench paper (RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts (Wijk et al., 2024))