Are we ready to navigate the complex ethics of advanced AI assistants?

An important new paper lays out the importance and complexities of ensuring increasingly advanced AI-based assistants are developed and used responsibly

Image: Midjourney

Last week a behemoth of a paper was released by AI researchers in academia and industry on the ethics of advanced AI assistants.

It’s one of the most comprehensive and thoughtful papers on developing transformative AI capabilities in socially responsible ways that I’ve read in a while. And it’s essential reading for anyone developing and deploying AI-based systems that act as assistants or agents — including many of the AI apps and platforms that are currently being explored in business, government, and education.

The paper — The Ethics of Advanced AI Assistants — is written by 57 co-authors representing researchers at Google Deep Mind, Google Research, Jigsaw, and a number of prominent universities that include Edinburgh University, the University of Oxford, and Delft University of Technology. Coming in at 274 pages this is a massive piece of work. And as the authors persuasively argue, it’s a critically important one at this point in AI development.

This importance comes from the profound impact that AI assistants are likely to have on society as they’re incorporated into everything from personal decisions and actions to education, business, government, the maintainence and growth of critical infrastructure, and more.

With a growing number of apps based on large language models — and more advanced ones coming (including rumors of a major upgrade to Apple’s Siri), we’re already moving rapidly toward a future where AI assistants are ubiquitous. As the authors point out, “AI assistants have the potential to be a profoundly impactful technology via their deep integration into almost every aspect of our lives.”

Here, advanced AI assistants are defined in the paper as “artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user – across one or more domains – in line with the user’s expectations.”

Considerable attention is paid in the paper to developing and justifying this working definition. A critical challenge faced by the authors was the distinction between a “moralized” definition — encompassing agents that address outcomes such as empowering users to make better choices related to their beliefs, worldview, or moral framework for instance — and a “non-moralized” definition — one that encompasses agents that perform administrative tasks on a users’ behalf.

This distinction may seem nit-picking, but it has considerable influence on how the ethics of AI assistants are addressed. Here, the authors wisely took the pragmatic approach of focusing on advaced agents designed to be of practical service to users rather than, say, acting as moral guides.

That said, the paper blurs the lines between moralized and non-moralized definitions at a number of points as it considers the potential impacts of advanced AI assistants in areas like wellbeing, trust, and human-machine relationships.

To emphasize this, chapter 4 considers four possible types of advanced AI assistants: A thought assistant for discovery and understanding; a creative assistant for generating ideas and content; a personal assistant for planning and action, and a more advanced personal assistant to further life goals. These are all practical applications, but they also intersect at some point with user values and beliefs.

If anything, the ambiguity here between moralized and non-moralized assistants underlines the magnitude of the challenge the authors took on — and the reality that we cannot simply codify AI ethics within a neat set of principles where the technology has profoundly complex and largely unknown consequences to society.

This is where the authors do an important job of mapping out a landscape that desperately needs further research. They effectively push the dialogue on AI ethics from rather limited conversations around responsible use of generative AI, to the challenges and opportunities associated with what emerges from incresingly powerful foundation models.

The argument here is that large language models in particular are leading toward general purpose AI platforms that use a very human-like natural language interface to assist individuals in many different ways. And as they do, this will lead to transformations across society — and ones that could go seriously wrong if we don’t learn how to navigate them effectively.

There’s a clear sense reading the paper that, even though organizations are rapidly building new capacity around existing generative AI, we have barely scratched the surface of how to ask the right questions around the ethical and responsible development of these capabilities.

Getting to specifics, the paper addresses 14 key areas associated with ethical and responsible development of AI assistants. These cover value alignment, well-being, safety, malicious uses, influence, anthropomorphism, appropriate relationships, trust, privacy, cooperation, misinformation, equity and access, economic impact, and the environment.

Across these, a comprehensive picture is developed of the potential challenges presented by advanced AI assistants that is rooted in solid scholarship. It’s a picture that is sophisticated in its breadth and impressive in its depth. And it’s one that provides a good model for any exploration of the ethics of AI.

It also illustrates just how complex the issues are that these emerging technologies are presenting society with.

Reading through the paper, I don’t agree with the authors on every point — for instance, I think there’s more nuance to questions around anthromorphism and AI than they indicate. Yet this is also very much the purpose of the paper — to stimulate serious discussion and research around questions that have prescious few answers at the moment.

As I mentioned at the outset, this is a long paper, and to try and summarize it would do it an injustice. Recognizing this, the authors provide a reading guide on page 16 — my appreciation for them grew in leaps and bounds when I came across it!

I’m actually going to replicate their reading guide in full here, as it’s a great guide to what is a must-read paper for those who don’t have time to digest the whole thing:

• 10-minute read: Read the ‘Key Questions’ in Chapter 1 alongside the ‘Key Themes and Insights’ from Chapter 20.

• 45-minute read: Read Chapter 1 and Chapter 20 alongside Chapter 19 on Evaluation.

• Readers with no background on LLMs: We recommend that you read Chapter 1 alongside Chapter 3 on Technical Foundations and Chapter 4 on Types of Assistant. These chapters provide an accessible introduction to LLMs and the techniques used to adapt them into advanced AI assistants. These chapters also provide the necessary technical foundation for understanding the ethical discussion that follows.

• Readers with an interest in technical AI safety: We recommend that you read Chapter 5 on Value Alignment, Chapter 7 on Safety, Chapter 8 on Malicious Uses and Chapter 19 on Evaluation.

• Readers with an interest in privacy, trust and security: We recommend that you read Chapter 8 on Malicious Uses, Chapter 12 on Trust, Chapter 13 on Privacy and Chapter 19 on Evaluation.

• Readers with an interest in human–computer interaction: We recommend that you read Chapter 9 on Influence, Chapter 10 on Anthropomorphism, Chapter 11 on Appropriate Relationships, Chapter 12 on Trust and Chapter 19 on Evaluation.

• Readers with an interest in multi–agent systems: We recommend that you read Chapter 14 on Cooperation. You may also want to read Chapter 5 on Value Alignment.

• Readers with an interest in governance and public policy: We recommend that you read Chapter 12 on Trust and Chapter 17 on Economic Impact. You may also want to read Chapter 15 on Equity and Access, Chapter 16 on Misinformation and Chapter 18 on Environmental Impact.

• Readers with an interest in philosophical foundations: We recommend that you read Chapter 2 on Definitions, Chapter 5 on Value Alignment and Chapter 6 on Well-being.

These highlighted areas are extremely useful in guiding readers through specific questions and challenges. What they don’t do is provide insights that are domain-specific.

Here, I was especially interested in how the paper addresses the use of AI assistants in education. This is a massive growth area, with an increasing number of companies persuading K-12 educators to adopt new tools based on AI assistants. Universities are also beginning to explore and develop everything from AI assistants to advise students to those that develop courses, construct syllabi, and grade assignments.

Much of this work is being carried out with an eye on ethical uses of AI. But I wonder whether the frameworks being employed are becoming increasingly disconnected from the challenges that advanced AI assistants present.

While most of the paper doesn’t address advanced AI assistants in education directly, it does incorporate educational applications within the broader discussion. The authors note, for instance, that “efforts are currently underway to develop advanced assistants that are able to function as personal planners, educational tutors, brainstorming partners, scientific research assistants, relationship counsellors and even companions or friends.” Many of these areas touch on apps that are currently being developed for or used in K-12 and beyond.

Chapter 17 looks more specifically at a case study exploring the potential economic impacts of educational assistants. In this chapter the focus is on the balance of potential positive versus adverse impacts in areas like employment (jury is out on job gains/losses, but AI assistants are likely to disrupt learning ecosystems); job quality (likely worse without institutional training and support); productivity growth (could potentially help improve the quality of education while helping students to use AI, leading to a significant boost in human capital and productivity); and inequity (potential to decrease through access to education).

The analysis here is definitely on the positive side when framed in terms of economic impact, which is encouraging. However, the use of advanced AI assistants in education also needs to be contextualized in the light of the rest of the paper.

Here, there are no cut and dried insights and conclusions. But it is clear that the authors see the ethical landscape around advanced AI assistants — including their use in education — as a complex balance between substantial opportunities and serious risks.

For instance, on the opportunities side they highlight empowering users, improving wellbeing, enhancing creativity, optimizing time, promoting user flourishing and autonomy, and supporting broader networks of human interaction and relationships (and I’m missing out a lot here).

On the other hand, they indicate advanced AI assistants have the potential to be misaligned with user and social interests, impose values on users, be used for malicious purposes, hinder self-actualization, increase emotional and material dependence, and more.

Overall the paper concludes with 62 opportunities, risks, and recommendations (starting on page 201). These are worth reading, even if you skip over the rest of the paper.

As a final reflection, I can’t emphasize enough how this paper moves us away from some of the potential short sightedness that’s set in around current generations of generative AI. In doing so it opens up pathways to explore the broader implications and opportuities of what comes next as we build AI systems that engage with us in very human ways, and have agency to change our lives — and even ourselves.

As the authors write at the end of their executive summary:

“We stand at the beginning of an era of technological and societal transformation marked by the development of advanced AI assistants. Which path the technology develops along is in large part a product of the choices we make now, whether as researchers, developers, policymakers and legislators or as members of the public. We hope that the research presented in this paper will function as a springboard for further coordination and cooperation to shape the kind of AI assistants we want to see in the world.”

It’s a strong yes from me to that!