Dario Amodei, CEO of Anthropic (one of the leading AI labs together with OpenAI, Google DeepMind, and Meta), published a long essay, Machines of Loving Grace, about “how AI could transform the world for the better.”
Amodei, like Anthropic, is outspoken about AI risks more than the benefits because he thinks the upside will be realized anyway, without us having to hype it (I agree). However, he says, we need collective action and a special kind of carefulness to navigate the dangerously turbulent waters of the AI alignment problem—how to align an AI smarter than the smarter human (or even all humans) with humanity’s values and goals.
That’s why this essay was particularly exciting to me compared to the writings of people well-known for their quasi-unconditional techno-optimism, like Sam Altman.
Before we start, it’s important to note a crucial caveat: Amodei’s entire essay—as well as this one, which is effectively a response—requires a leap of faith, or, at least, a leap in time.
He talks about the future of AI, particularly about powerful AI (that’s the name he gives to what people commonly call AGI, artificial general intelligence) but doesn’t describe the steps we would need to get there from where we are today.
He doesn’t say because he doesn’t know. No one does.
He admits to thinking powerful AI is close (perhaps this decade) but concedes there’s some intrinsic uncertainty and several hurdles that could delay the timelines (he even writes the word “if”, implicitly ascribing a non-zero probability to the possibility that we never get there).
So he takes a leap (of faith or time, depending on what you believe about AGI) and starts his exposition from a standpoint that invites us to assume his two premises (I accept them to write a reasonable response despite not necessarily accepting them beyond this post):
We will manage to solvent the risks of AI, especially the alignment problem.
We will achieve powerful AI so that it is equivalent, using his analogy, to a “country of geniuses in a datacenter.”
Once I convinced myself to set my response into his chronologically undefined future, I realized I concord with most of his optimistic predictions for the 5-10 years after powerful AI is achieved (what he calls the “compressed 21st-century”), across disciplines like biotech, neuroscience, health, economic growth, work, and even democracy. I bet you do, too. Here’s a sample:
AI could cure most physical and mental diseases and reach “biological and cognitive freedom” (which includes the ability to alter our anatomy, genes, and brain connections).
AI could invent new techniques, improve current ones, discover new knowledge, and propose ideas no human could come up with on their own. Imagine a general-relativity-level intuition every few months.
AI could bring about an unprecedented period of economic growth, productivity, and prosperity—beyond the Industrial Revolution—potentially leveling to some degree the developed and developing worlds.
AI could materialize Francis Fukuyama’s dream and end history by bringing liberal democracy—and peace as a consequence—to every corner of the world.
AI could render work unnecessary and suggest alternatives (including but not limited to universal basic income) while allowing us to find meaning elsewhere.
Most of those sound—albeit bordering on science fiction, references to which he intentionally avoided—like things we’d want. And I hope you agree with that (as long as you allow yourself to be dispassionately detached, as a thought experiment, from the customs of the current world).
However, for all the optimistic prospects he lays out, I’m surprised he never mentions or even hints at—which makes me suspect he may not have considered it—the one thing that seems to me like a nightmarish outcome, despite everything else going right. I’m not talking about the risks of AI—remember we managed to overcome those—but about the risk of human identity.
The questions of who we are, why we’re here, and where we’re headed have always been painfully elusive philosophical dilemmas. But powerful AI will reveal the harsh truth: we are unworthy recipients of such arrogantly self-important questions.
Now, I want to be very clear on what I mean by this because talking in the abstract allows ambiguity to leak into my words. First what I don’t mean. I don’t think AI will become more human than us and thus we’ll fail at that. No, that’s not a thing—we’ll always have a competitive—not even comparative—advantage in being better at being human. I also don’t think we will have to face the trouble of interacting with and relating to an alien species. We may live in the analog world while AI, albeit powerful, remains sealed and isolated in its “datacenter-as-country”.
What I mean is this: I’m not sure humanity can afford—mentally, spiritually, or identitarily—to be at the mercy of an unfathomable group of artificial geniuses. A group capable of enacting progress beyond our reach, but at the cost of reducing us from active participants to passive spectators. From travellers of the cosmos to mere cosmic luggage.