Audience at a Magic Show

This is what the future holds

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (back soon). If you’d like to become a paid sub, here’s a button for that:

This is one of those times when I feel like I’m the only person who’s not blind.

Roon, an always optimist but also sensitive OpenAI staffer, urges us not to “be reduced to the status of audience at a magic show.”

Don’t stand there, he says, clapping your monkey hands. These mysteries are for you to grasp. Want to learn the magic trick? Want to take a peek at the show’s behind the scenes? Well, maybe grow 200 IQ points then.

He is, naturally, talking about OpenAI’s barrage of mathematical results.

It’s a document with over 700 math solutions—in varying stages of the verification process—many of them, if proven correct, of great historical importance; some of them even more than Navier-Stokes.

No, that’s still an understatement. Try this: If a human did something like this, they’d be ipso facto crowned “God of Numbers.”

That’s how big this is.

The conquest of math by AI has been going for months now but today is different. I’ve seen serious people say that, together, these results are the biggest thing ever to happen in mathematical history. That it’s the equivalent to a nuclear bomb for mathematicians. No hesitation or qualification whatsoever.

There are a bunch of Fields Medals hidden in the document. Or rather, the Fields Medal doesn’t make sense anymore because of that document. What sort of advance can render the main recognition of an entire discipline meaningless? This one.

But I’m not bitter. After all, I wouldn’t have been able to grasp these results had it been a human who solved them instead of a machine. To me, it amounts to the same, for I’ve been mere audience to a magic show all my life. It’s just that it feels weird that the magicians I used to look up to are now applauding next to me and it’s mathematics itself that’s doing the magic.

That is weird.

I said that roon is optimistic but he’s actually coping hard. He’s too smart—and too much of an insider—to start at the denial stage so he went straight to the bargaining stage. He says that the magic is for us to understand, but is it? Not only is the amount of stuff to go through insane or the speed at which it happened appropriately superhuman, but the complexity itself is bordering on the limits of what the best of us are capable of. Guys who spent decades in some of those problems are awestruck. Never before has the expression “being struck with awe” made so much sense.

And so I’m reminded, once more, of chess and Go.

Because whatever it is that AI conquers next, chess and Go already went through it: it’s “move 37” and whatnot all over again! And what was the conclusion then? That humans need not apply. We can’t keep up. The set of “best players of all time” couldn’t keep up. They admitedly learned quite a few tricks from the AIs, of course, and they even surpassed the limits we thought possible for an ape with an overgrown soup of neurons inside the skull. Amazing! But that’s it. Not only can Magnus Carlsen never ever win against Stockfish, he can’t even understand what it does or why.

It’s simple: learning from superhuman intelligence is not the same thing as remaining intellectually commensurate with it.

So why does roon, supposedly Bitter Lesson-pilled, imagine it will be different for math? Or physics or biology or whatever they decide to throw into the machine in a trade-off that can only be described as “progress despite us”? It won’t be different. We still see Carlsen’s magic shows because watching an ape do chess is fun. But math? Nah just show me those impressive results and I’ll applaud.

What roon and his colleagues in the AI industry are doing is, effectively, passing the baton to AI on behalf of humanity. We will no longer be proud cosmic wanderers but timid cosmic luggage—or worse, cosmic tourists. We are, certainly, no longer magicians at the show but awestruck audience at best.

I argued, when I first wrote about what would happen when they started indiscriminately pointing the AI gun at the math targets, that it would be a problem trying to make sense of all these incredible results. Insofar as we wanted to use them for what is, by all accounts, still a human world, we’d need a connective tissue. Someone who could receive the gift of magic and translate into material gains or knowledge. After all, the world can only make sense of a discovery if the intelligence of the receiver—not the discoverer—is enough to make sense of it.

Who will play the role of connective tissue, I asked. Who will go down into the machines’ dressing rooms to discover the meaning of their magic and then make it legible for us, inferior animals? Only a few can, I thought.

But naive as I was—or perhaps only hopeful—I made two serious mistakes.

First, thinking that there would always be someone who could. And second—even worse—assuming that such a role would continue to be necessary. I thought, and even said explicitly: the only scenario where it won’t be necessary is if this world ceased to be human. Of course, I reasoned, that will never happen. I realize I didn’t pursue that line further not because I believed it was impossible, but to protect my emotional well-being.

Well, it’s too late for that, roon.

I understand your intention—and praise it—as well as the psychological drive that makes you write that tweet. But you’re deeply wrong. You—and many like you—are directly responsible for the reality you’re now refusing to accept. Actually, I don’t know if you are coping or just gaslighting us. You seemingly want us humans—proud, adaptable humans—to maintain the subject-object relationship with the intellectual fields that the AI you make is destroying. It’s not destroying the disciplines themselves, to be sure, but the relationship we had with them. AI might kill us all, who knows, but now I wonder if it won’t even have the chance because we’ll all die first of sadness.

Forgive my pessimism here. I just can see into the future. I can see where this leads us. And I see we have no means of stopping it.

So dare follow the logic to its endpoint, roon, and dare face the consequences of your actions. There’s no hiding behind vain hopes anymore. Even you, at the epicenter of what is perhaps the most important transformation this world has experienced, can’t accept it. You’re an actor in the last human show, but I don’t feel like applauding.

I guess it’s true that “courage is in shorter supply than genius.” There isn’t any more courage in trying to reach the stars than in admitting they are, perhaps, too distant.

And yet, as you build a stairway to the stars with this magic of yours, you encourage the human to jump, like the ape he is, to reach just as far. If he only jumped high enough, you say, he may brush the rim of the firmament. Just a bit higher… And in doing so you pretend this truth doesn’t need reconciliation: in making this world more magical—what a time to be alive!—you’re also making it less human.