Will Elon Musk's "Maximally Curious" AI really turn out to be safe?

Musk's new company x.AI aims to create an artificial general intelligence based on curiosity and truth seeking. What could possibly go wrong?

Image: Midjourney

I have to confess that, for all his many flaws, I can’t help but be inspired sometimes by the audacity of Elon Musk’s vision. I was certainly feeling this last week as Musk launched his latest venture, x.AI on Twitter Spaces. But there was something that bugged me about the the ideas underpinning the company — so much so that I went back and re-listened to the whole two hour launch event to check that I’d heard things right.

x.AI certainly has an audacious goal: to “understand the true nature of the universe.” The company will attempt to achieve this by building a super-powerful artificial intelligence that, they hope, will emerge as an artificial general intelligence (AGI) that can solve some of the biggest challenges of life, the universe, and everything. And yes, there is a very “Douglas Adams” vibe to the company for those of you who spotted the Hitchhikers Guide to the Galaxy connection.

This, I must admit, is exciting stuff. As a physicist and generally curious person, I love the idea of working with intelligent machines to unravel the mysteries of the universe. But there was something that didn’t sit well with me when it came to the underlying philosophy guiding this very Musk-esque enterprise, and especially in how the company are planning to ensure they create a benevolent AGI rather than one that panders to fears of an AI apocalypse.

Of course, Musk has been very vocal over the years on the potential risks that AI presents, and the need to ensure the safety of emerging AI and AGI. He’s a firm believer that superintelligent machines could present an existential risk if we get things wrong. At the same time, he believes that the emergence of superintelligent AGI is inevitable — and so he sees it as a moral and social responsibility to actively work toward ensuring that it enhances rather than diminishes humankind.

The only problem is that Musk’s ideas of how to achieve this draw on his own unique brand of futurism. And here, Musk argues that it’s actually dangerous to engineer-in morals and ethics to AGIs as a way of ensuring they are incapable of causing harm to humans.

To Engineer Moral Machines or Not …

The idea of engineering moral guardrails into artificial intelligence is, of course, one that goes back at least as far as Asimov’s Three Laws of Robotics, and is reflected in discussions around “value alignment” in AI where machines align with and adhere to human values and goals.

Value alignment is central to most approaches to long term AI safety. But it has its problems as that there’s no universal consensus on what an appropriate set of values or morals for AGI might look like. Even if there was, it’s by no means certain that it would be ethical to constrain an AGI by hardwiring its understanding of right and wrong (although this in itself is a contentious issue).

Beyond these concerns though, Musk argues that hard-coding morals into AGI could potentially lead to what he calls the “inverse morality” problem. This is a hypothetical scenario where there’s a risk that creating a “moral” AGI will naturally lead to an immoral counterpart emerging. It’s an idea that’s somewhat more complex (and more tenuous) than the Super Mario Bros analogy Musk used in the x.AI launch (the quote was “If you make Luigi, you risk making Waluigi at the same time”). However, it is an idea that is clearly guiding how the company is approaching AGI.

Instead, Musk argues that we should be building AGI that comes to the conclusion on its own that humanity is worth nurturing and valuing — paving the way to a future where we live in harmony with superintelligent machines.

This is an idea that has deep resonances in science fiction — most notably perhaps, in Musk’s case, with the writer Iain M. Banks and his Culture series of novels (Musk is a great fan of Banks’ work). In Banks’ novels, machines that are vastly more intelligent and powerful than humans cherish what makes biological life forms unique and interesting, and as a result they form an integral part of a complex galactic society.

Interestingly, this idea of learning what’s right and wrong, rather than being constrained to a pattern of behavior, also has religious overtones — remember the importance that’s placed on free will and choice in the story of Adam and Eve? That said, I’m not sure that this is a connection is top of x.AI’s mind.

Will Humanity’s Uniqueness Lead to future AIs Protecting Us?

Of course, Musk’s arguments for not hard-wiring morality into AGI are grounded on more than sci-fi novels. Importantly, he suggests that because there’s a reasonable likelihood (in his estimation) that human-like intelligence is a supremely rare thing — and possibly unique to Earth — there’s a good chance that, to a smart AGI, “humanity is much more interesting than not-humanity.”

In other words, if an AGI grew to realize just how precious and unique humanity, together with human consciousness and intelligence, are, it would ideally come to the conclusion that we’re worth protecting and nurturing.

And the way that this will happen — according to Musk and the x.AI team — is through creating AGI that is driven by “maximal curiosity and truth seeking.”

Effectively, Musk argues that if an AGI is driven by curiosity and discovering the truth, it will naturally prioritize humanity’s interests because, in its curiosity-driven search for the truth, it will discover just how special we are!

Putting aside the high stakes gamble that we do indeed turn out to be the apple of some future AGI’s eye, this is a philosophy that’s fraught with poblems.

For starters, I’m not sure there is a strong causal link between curiosity and benevolence. Rather, in the absence of an underlying moral framework, the opposite may be more likely. How many scientists have slipped into ethically reprehensible behavior, simply because the moral guardrails around their “what if?” questions were inadequate to contain the extent of their curiosity? And how many young children have dismembered and mutilated living things, just because they were curious about what would happen of they did …

Ultimate Truths, Blinkered Beliefs

Then there’s the “truth seeking” dimension to x.AI’s ambitions. In the company launch, Musk — and many of his team — came across as believing that there are ultimate, indisputable truths that govern the universe we live in, the world we live on, and the society we live within — and that the secret to humanity’s future is to discover these truths and to combat untruths, especially when these are ideologically or politically motivated.

The existence of ultimate truths in the context of physics is an interesting philosophical question, and one that’s far from certain (although I would tend toward Musk’s deterministic view of things here). But we know very well that the “truth” underlying the physics of the universe cannot simply be extended to a black and while understanding of “truth” when it comes to people, society, and our complex relationships with each other and the environments we inhabit.

In fact an assumption of ultimate social “truth” is more often an excuse to impose an ideology or worldview on others while diminishing or erasing their own “truth.”

This, of course, is an age-old debate around “the truth” versus “truths,” together the deterministic nature — or otherwise — of humanity. However, most scholars I work with would either be deeply skeptical of the notion of “ultimate truth” when it comes to people and society, or completely dismissive of the idea.

As a result, this foundation of “truth seeking” worries me. It’s a “truth at any cost” attitude that is more likely to fuel deeply divisive initiatives that are driven by what a small group of people — or machines that have been coded by them — determine to be “truth.”

In other words, being driven by “truth seeking” — where you get to decide what constitutes “truth” — becomes its own version of the “inverse morality problem,” where an assumption that truth seeking leads to a more moral society ends up having the opposite effect.

That said, there is a kernel of sense in x.AI’s approach. In today’s hyper-interconnected and social media-mediated world, it’s becoming increasingly difficult to tease out evidence-grounded and socially responsible ideas and perspectives from harmful ideologies and political obfuscation and manipulation. And so it’s not hard to see how a “truth seeking” AGI might be imagined as helping cut through the noise.

This is where, in the x.AI launch, Musk explicitly highlighted the dangers of optimizing AI for what he referred to as political correctness — essentially teaching it to favor what is expedient over what is true, and to “lie” to protect ideas, ideologies, and agendas.

I have some sympathy with concerns here — not necessarily with political correctness per se, but in who determines what an AI’s agenda is; and how this impacts its actions, its trustworthiness, and — as we potentially move toward sentient AI — its very sanity. And I’m not sure we want AIs that are as slippery with the truth as politicians and preachers of ideologies.

And yet, if we strip away any semblance of subtlety and complexity around what “truth” means in AGIs, along with the messiness and ambiguity that so often accompanies trying to make sense of a world that defies sense-making, we risk creating machines that are deeply inhuman.

And in this case, a maximal curiosity and truth-seeking AGI probably isn’t going to do humanity a whole lot of good.

A Soupçon of Optimism

This is where I suspect that, if x.AI does succeed in creating their curious, truth seeking artificial intelligence, it will open up a whole new can of safety worms that we’re ill-prepared to handle. And yet despite this, I’m still intrigued to see what x.AI will achieve — blame it on my curiosity! And I am hopeful that the company will learn to listen to others, to be humble, and to grow out of its over-simplistic understanding of how the world works as it builds the next generation of AI.

Because beyond the challenges that we face in achieving increasingly advanced artificial intelligence, the vision of creating thinking machines that can work with us to explore the universe and create a vibrant future together is, when all is said and done, pretty inspiring.