Transformative AI

Transformative Artificial Intelligence

Progress in the field of artificial intelligence (AI) has been extraordinarily rapid in recent years. Although the future impacts of AI remain highly uncertain and are subject to deep disagreement, there is a very real possibility that further advances in the field could lead to radical economic and social transformations and that navigating these changes could be among the most significant challenges we face in coming decades. Part of our research therefore aims to inform our understanding of the potential for transformative societal changes due to the development of advanced AI systems, the largest-scale challenges likely to arise as a result, and the best means of addressing them.

Catastrophic Risk

It has been argued that advanced AI systems pose significant catastrophic risks, up to and including the risk of existential catastrophe (Yudkowsky 2008; Bostrom 2014; Russell 2019; Cotra 2022; Carlsmith 2025; Hendrycks 2025; Kulveit et al. 2025). Other large-scale risks include extreme power concentration and democratic collapse (Drago and Laine 2025; Acemoglu, Gitmez, and Shadmehr 2026). Many arguments for catastrophic risk appear to turn on philosophical assumptions, such as instrumental convergence (Omohundro 2008; Bostrom 2014: 105-114; Gallow 2025). How should we approach and assess these arguments? Which threat models are worth taking seriously, and how should we prioritize among the different catastrophic risks they spotlight? (Note that here – as elsewhere – we assume no particular conclusions: we are interested in work arguing that risks from AI are overstated, as well as work that might strengthen the case for catastrophic risk.)

Many have argued that one of our key strategies for mitigating catastrophic risks associated with the potential for loss-of-control should be to ensure that AI systems are aligned with human values. However, there remains deep disagreement about how best to conceptualize alignment: in particular, to what extent alignment should be understood in terms of deferring to human preferences and commands as opposed to the internalization of appropriate ethical values capable of guiding autonomous choice (Leike et al. 2018; Gabriel 2020; Askell et al. 2021; Zhi-Xuan 2025). A wide range of further technical and governance-based mitigation strategies have been proposed for addressing catastrophic risks from AI (Dafoe 2018; D’Alessandro and Kirk-Giannini 2025). What are their strengths and weaknesses? What novel mitigation strategies remain to be analysed in depth?

The Trajectory of Civilisation

Even setting aside the possibility of catastrophe, the development of transformative artificial intelligence could potentially bring about profound societal changes, including extremely rapid economic growth (Trammell and Korinek 2023; Davidson et al. 2026) and large-scale technological unemployment (Susskind 2020; Korinek and Juelfs 2024). Social norms and institutions may be upended due to the rapid pace of change, before potentially crystallizing into some new, highly durable configuration (MacAskill 2022: 75-102).

Ideally, we’d like to know not only how plausible it is that advanced AI might bring about different possible societal transformations, but also how best to navigate the different risks and opportunities this might pose. This may include the desirability and feasibility of broad-based improvements in the ability of key societal institutions to respond to rapid changes, as well as preparatory efforts to map out key challenges and best responses beforehand, reducing the risk that we’ll be caught off-guard (MacAskill and Moorhouse 2025). Last but not least, it may involve efforts to map out realistic, positive visions of a society transformed by the impacts of advanced machine intelligence, providing a sense of what to aim and hope for.

Digital Minds

The topics outlined above are most naturally thought of as concerned with the potential positive and negative impacts of advanced AI on human flourishing. Impacts from transformative AI will stretch far beyond the human realm, however. Non-human animals will be impacted in myriad ways, such as through advances in precision livestock farming (Simoneau-Gilbert and Birch 2024). AI systems may themselves become moral patients, meriting our concern and respect (Long et al. 2024). We are interested in research at the intersection of ethics and the philosophy of mind and cognitive science relevant to understanding what kind of moral standing, if any, AI systems could have, and how we should respond to different forms of evidence relevant to settling that question. For more details, see here.