Why the recent statement on the risk of extinction from AI is important, and why I didn't sign it

A new statement from the Center for AI Safety claims the extinction risk presented by AI should be a global priority. I argue that potential catastrophic risks should be a higher priority

Image: Midjourney

A statement from the Center for AI Safety released on Tuesday this week caused something of a stir, and further amplified intense discussions around the risks of artificial intelligence.

The statement is a very simple one:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

What makes it particularly interesting is the list of signatories. These include a number of highly respected thinkers, scholars, and innovators, including Yoshua Bengio, David Chalmers, Martin Reese, Andrew Revkin, Stuart Russel, Sam Altman, and many others.

These are intelligent people and deep thinkers — in many cases they are experts who have been at the forefront of sophisticated thinking around the intersection between people, technology, and the future. And because of this, it’s worth pausing to consider the statement rather than dismissing it as merely fear mongering or naive speculation.

I certainly agree with the underlying sentiment behind the statement. Emerging AI capabilities, from foundation models to interfaces like CharGPT, are raising early warnings around the likely societal disruptions that future iterations of AI could bring about. Many of these are likely to depend less on intelligent systems per se, and more on powerfully and unexpectedly disruptive systems that are deployed with little understanding of possible consequences.

Here, it’s easy to frame the risks of AI in relatively conventional ways, seeing potential disruption in terms of areas like educational challenges, job displacement, automation, privacy, etc. These are all important. But I suspect that they’re the shavings off the tip of the AI iceberg when it comes to future impacts. They also run the danger assuming that the risks associated with a highly unconventional technology transition like the one represented by AI can be sliced, diced, and solved, using a conventional mindset.

Based on all my years of working on the risks and benefits of emerging technologies, this strikes me as being extremely naive.

Rather than these important but probably tractable “conventional” risks, what concerns me more is the possible threats to social, economic, and political structures, systems, and norms. Here, there is a small but finite chance that emerging AI will profoundly impact the ways we interact with one another, the ways we individually and collectively build our understanding of the world we live in and the people we live with, the ways we live our lives and make decisions, and the underlying mechanisms that keep society going.

These concerns are grounded in many different aspects of emerging AI, and they include the seductive mastery of language being exhibited by large language models, the interference of AI in decision making and autonomy, the disruption of democratic processes, and even the modulation by AI systems of the flow of everything from goods and ideas to information and misinformation.

These and more are all part of a highly non-linear technology transition that cuts across highly integrated complex systems, where it’s the seemingly trivial and unexpected that are likely to lead to potentially catastrophic outcomes.

Of course, they also may not — people are incredibly good at finding innovative solutions to complex challenges on the fly, and it may be that we successfully use our wit, agility, and problem-solving ability as a species to create a vibrant AI future.

The problem is, there is no guarantee of this — and usually where the stakes are high and the future is uncertain, it pays to be cautious.

Because of this, I’m a strong proponent of the deep and broad intellectual explorations that are needed to ensure the emergence of beneficial AI while avoiding potentially catastrophic failures. These, though necessity, need to draw on many different areas of understanding and expertise, as well as engaging a broad range of stakeholders. They also need to consider the possibility of failure modes that are as far from conventional as you can get — alongside approaches to navigating them that are equally unconventional.

All of this would seem to align with the statement from the Center for AI Safety — and as you can probably guess, I have a lot of sympathy with the statement. And yet, I baulk at the framing around the “risk of extinction.”

This, to me, is both too narrow and absolute a framing, and too human-centric. We need to be thinking more holistically than the end of the human race — we are, after all, an integral part of a larger set of interconnected ecosystems. We also need to recognize that, while extinction is a vanishingly small possibility (it’s hard to imagine a future where every last speck of humanity has ceased to exist), the possibility of catastrophic risk is not so small.

AI-induced catastrophic risk is far more likely that extinction — and far more worrisome. Catastrophic risk is typically defined as the risk of an event where large numbers of people are severely impacted. I would extend this based on our work around risk innovation to events and emerging circumstances where large numbers of people risk losing something that is deeply valuable to them — whether this is their life, their health, their dignity, autonomy, purpose, family, or many other types of “value” that are critical to our individual and collective lives.

Framing the potential risks of AI in this way highlights the deep complexity of the risk-benefit landscape we are facing, and the new thinking that will be needed in order to successfully navigate it.

I suspect that many of the signatories of the Center for AI Safety statement would agree with this. I am concerned though that, by framing the debate around risk in terms of extinction, much of this nuance will be lost.

Interestingly, this becomes important when the potential catastrophic risks of not developing new AI-based technologies are factored in to the equation. By framing catastrophic risk as the potential loss of something (or things) of value at scale, we can also consider the potential loss of solutions to pressing challenges — climate change, poverty, equity, threats to democracy, and more. Where AI may provide unique pathways forward that are unlikely to emerge without it, AI as solution has to be part of the discourse around AI and risk.

The bottom line is that AI is likely to be one of the most potent technological disruptors we’ve encountered for a very long time, and it would be foolish not to be concerned about existential-level risks, or to begin exploring ways of avoiding them. I would go so far as to agree with the Center for AI Safety statement that the potential risks of AI should absolutely be a global priority, and a central pillar of any initiative focused on ensuring a positive global future.

At the same time, there needs to be a framing of AI risks and benefits that opens up new possibilities rather than closing down conversations.

This is where I hope that we can get beyond speculations around extinction-level events and accusations of “AI-doomsaying,” and get on with the challenge of approaching potentially catastrophic AI risks with the respect, due diligence, and intellectual rigor and imagination, that they deserve.