Nick Bostrom is a Swedish-born philosopher who founded the Future of Humanity Institute (FHI) at the University of Oxford, where he served as founding director from 2005 until its closure in 2024. His 2014 book Superintelligence: Paths, Dangers, Strategies is one of the most cited works in AI safety and helped bring existential risk from artificial intelligence into mainstream academic and policy discourse.
Background
Bostrom's research at FHI framed the problem of advanced AI in terms of control and value alignment, arguing that a sufficiently capable AI system pursuing misspecified goals could pose an existential threat. His concept of Recursive Self-Improvement (RSI) — an AI rapidly improving its own capabilities beyond human ability to intervene — became a foundational concern in safety research, and his work shaped public discourse on when AGI might arrive.
By his own account, Bostrom has an academic background spanning theoretical physics, computational neuroscience, logic, and artificial intelligence, in addition to philosophy, and he describes himself as one of the most-cited philosophers in the world (Source: https://nickbostrom.com/). He received an undergraduate degree at the University of Gothenburg in philosophy, mathematics, mathematical logic, and artificial intelligence, and subsequently pursued postgraduate degrees in philosophy, physics, and computational neuroscience before arriving at Oxford as a postdoctoral fellow at the Faculty of Philosophy (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). He was involved in transhumanist communities, including the Extropians listserv, from the 1990s (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute).
Roles
Bostrom created and led the Future of Humanity Institute, which he established at Oxford in 2005 with seed funding drawn from a benefaction by the IT entrepreneur and futurist James Martin; the institute began with a small number of researchers and grew to roughly fifty staff at its peak (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute) (Source: https://nickbostrom.com/). FHI's research agenda began with the ethics of human enhancement and broadened over time into existential risk, AI safety, AI governance, biosecurity, and the moral status of digital minds (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). According to the Asterisk Magazine account, FHI helped incubate or shape several adjacent communities and organizations, including early empirical work on AI alignment, the Centre for the Governance of AI (spun out of FHI), the AI Impacts project, and the effective altruism and rationalist communities (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). Researchers associated with FHI included Anders Sandberg, Toby Ord, Eric Drexler, and, for a period, Jan Leike, who later worked on reinforcement learning from human feedback and led alignment work at OpenAI and Anthropic (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute).
After FHI closed, Bostrom became founder and principal researcher of the Macrostrategy Research Initiative, a nonprofit (Source: https://nickbostrom.com/). He describes his research as oriented toward understanding what he calls humanity's "macrostrategic situation" — the larger context in which civilization exists and how present choices relate to long-term outcomes (Source: https://nickbostrom.com/). He is the author of around 200 publications, including Anthropic Bias (2002), the edited volume Global Catastrophic Risks (2008), Human Enhancement (2009), Superintelligence: Paths, Dangers, Strategies (2014), and Deep Utopia: Life and Meaning in a Solved World (2024) (Source: https://nickbostrom.com/).
Concepts and publications
Across his writing Bostrom originated or developed a number of concepts that recur in AI-policy and existential-risk discussion, including the simulation argument, existential risk, the vulnerable world hypothesis, differential technological development, information hazards, astronomical waste, the unilateralist's curse, the notion of a singleton, and the moral status of digital minds (Source: https://nickbostrom.com/). His 2003 paper "Are You Living in a Computer Simulation?", published in Philosophical Quarterly, argued that at least one of three propositions holds: that the human species goes extinct before reaching a "posthuman" stage, that such civilizations almost never run ancestor simulations, or that we are almost certainly living in a simulation (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism) (Source: https://nickbostrom.com/). In "The Superintelligent Will," Bostrom set out the orthogonality thesis and the instrumental convergence thesis, which describe the possible range of behavior of advanced AI agents and some associated dangers (Source: https://nickbostrom.com/).
Superintelligence (2014), which Bostrom developed from a chapter of an earlier book on catastrophic risk, argued that the creation of machine intelligence surpassing humans could be consequential for the future of the species. The book was a New York Times bestseller and was publicly praised by figures including Bill Gates and Elon Musk, the latter of whom later became a donor to FHI (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute) (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism). In October 2015 Bostrom briefed a United Nations committee on risks posed by future technologies (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). His most recent book, Deep Utopia (2024), examines what life and meaning might look like in a world in which advanced AI has resolved most practical problems (Source: https://nickbostrom.com/) (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). On his website Bostrom has more recently circulated working papers on AGI governance, including "Open Global Investment as a Governance Model for AGI" and "Optimal Timing for Superintelligence," and states he is currently working on topics related to AGI governance (Source: https://nickbostrom.com/).
Closure of the Future of Humanity Institute
FHI was closed by the University of Oxford on 16 April 2024, after nearly 19 years (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute) (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism). The institute and Bostrom attributed the closure to administrative tensions with Oxford's Faculty of Philosophy; according to FHI's final report, the faculty imposed a freeze on fundraising and hiring beginning in 2020, and in late 2023 decided not to renew the contracts of remaining FHI staff (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism). Oxford's public statement said only that, after considering the best structures for its academic research, it had decided to close the institute and recognized its contribution to an emerging field (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). The Asterisk Magazine account reports a longer record of friction over bureaucracy, hiring, and the mismatch between FHI's output and the faculty's emphasis on peer-reviewed publication (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute).
Positions and reception
Bostrom has advocated transhumanism, the use of advanced technologies to extend longevity and enhance cognition (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism). On his website he states that his actual overall views are more nuanced and tentative than individual papers suggest, and rejects characterizations of himself as a "gung-ho transhumanist," as anti-AI, or as a strong ideological consequentialist, saying many of his papers deliberately analyze isolated aspects of a problem under specified assumptions (Source: https://nickbostrom.com/).
In January 2023, Bostrom published an apology for a message he had sent to the Extropians listserv in 1996, when he was a graduate student, after the message was circulated by the longtermism critic Émile P. Torres; in the message Bostrom had used a racial slur and asserted that white people were more intelligent than Black people (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism) (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). In the apology Bostrom wrote that he "completely repudiate[d]" the email (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). The University of Oxford suspended Bostrom and investigated; in August 2023 it stated that it did not consider him to be a racist or to hold racist views and that it regarded the apology as sincere (Source: https://asteriskmag.com/issues/08/looking-back-at-the-future-of-humanity-institute). Critics, including Torres, argued that the apology did not withdraw the underlying claim about race and intelligence and characterized longtermism as connected to eugenics; Bostrom subsequently said he had no particular interest in the question of race and intelligence and would leave it to others with more relevant expertise (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism).
Bostrom's framing of AI risk has been influential among policymakers and philanthropists who fund AI safety work; FHI received support from donors including the Open Philanthropy Project (backed by Dustin Moskovitz) and Elon Musk (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism). His emphasis on speculative long-term risk has been critiqued for prioritizing distant scenarios over near-term harms, and longtermism and effective altruism, movements associated with FHI, drew criticism following the 2022 collapse of Sam Bankman-Fried's FTX (Source: https://www.theguardian.com/technology/2024/apr/28/nick-bostrom-controversial-future-of-humanity-institute-closure-longtermism-affective-altruism).
Relationships
- related: AGI Timelines — his work shaped public discourse on when AGI might arrive
- depends-on: Recursive Self-Improvement (RSI) — central concept in his threat model