Fourteen-odd billion years ago, God snapped His large fingers and created the universe from nothing. In a time lapse so small it couldn’t be measured, the Big Bang expanded furiously, turning a soup of primordial particles into suns and galaxies. Earth emerged from stardust and the first life forms from stray water drops. Bacteria evolved into great apes—with branch-offs like octopi and cockroaches—to which we belong. Our brains got bigger, so we stepped down from trees. Our posture was bad, so the cosmic doctor advised us to walk erect. Finally, we invented fire and other useful stuff like guns and candles and it became pretty clear that we had the spark. It’s been a long way, with ups and downs—like that one time we invented agriculture—but God deigned to grant our plea nevertheless: “What if I put some thinking into their brains?” He thought. Whereupon He bestowed on us enough intelligence to explore the more interesting question: What if we take the brain out of our thinking?

As you might have noticed from the all-time low PISA scores, the fall of award-winning literature, or the fact that starting this year, no human will seriously consider a career in math ever again, we’re on a quest to do just that. The late sir Ken Robinson, expert in education, used to say that schools kill creativity—for that reason everyone’s tiktoks look the same, by the way—and so we have no choice but reach out to the most unspoiled minds of all: those of the youngest kids.

Paul Shearer’s seven-year-old daughter’s “Put some brain in your thinking” is up there in the all-time great aphorisms list with “I think, therefore I am” and “Know thyself.” It took a little girl to write the best six-word story since Hemingway, one that captures both the malady of our time and its solution; “Put some brain in your thinking” is as terrifying as “For sale: baby shoes, never worn,” but actually useful.

More credit to this budding philosopher for inventing the sentence. Google doesn’t register it. The very possibility of diagnosing the world with a brand-new coinage suggests the dawn of an era: even if its constituent concepts existed before—not thinking much (a dumb person), or lacking a brain (a dead person)—the synthesis (thinking without a brain) is unprecedented.

She’s not just a literary genius but, more importantly, native to the fact that brain and thinking are now disconnected. Substrate and activity go naturally hand-in-hand for us adults, but not for a member of Generation Alpha. There’s an interesting mix at play between pre-socialized innocence and post-AI birth. To us, ChatGPT is recent and the brain inextricable; to her, both have essentially existed forever and so they’re swappable ingredients of the thinking recipe.

She sees a truth that eludes our mature gaze: that although thinking happens by default, the brain is now an option among many in a growing catalog. Notice her syntax: put some brain in your thinking is to your eye a grammatical mistake because brain and thinking are synonymous. To her, however, this isn’t a clever inversion but natural speech.

In a world where thinking and brain are LEGO bricks, adding the brain back into the thinking is not an ontological argument about the human mind but pure practical advice. Like one says a salad could use more salt and pepper.

It’s in this sense that the brain has become a luxury input; an expensive spice you’d have to smuggle out of Arrakis. And so, just like tasty food and unsavory food are both food, the brain and AI are both thinking. Naturally, as a result, people will tend to overseason their thinking with AI. But unlike salt and pepper, the brain expires: they will carry it around as one does a vestigial organ. They might disengage their brains from life and work, but they can’t just take it off. They will become a forced cyborg and AI, a wheelchair for their shallow minds.

The fact that this split condition between brain and thinking is a novelty makes it all the more surprising that it’s also common.

I can stand a brainless human, but I hate to be a peerless one. ChatGPT follows me wherever I go and now you will, too? At least you flatter me by contrast: My skull does not house an NVIDIA H100 SXM5 80 GB HBM3 989 TFLOPs, but a cabbage made of neurons. My brain feeds my thinking and in turn I feed it by reading books, essays, and papers. Like this one by Graphite: “the number of articles published on the internet that are primarily AI-generated (50%) is equal to the number written by humans (50%).” The internet isn’t dead but you are definitely torturing it, hemorrhaging humanity out of it prompt by prompt.

These considerations are matters of vanity, however, and we have no time for that; when an isolated malaise turns into a pandemic, we must to ask a different question: can we undo it?

Yes, kind of. Cognitive atrophy is reversible like attention is reversible: it requires itself as the engine of reversion. It’s hard to stop scrolling because scrolling kills its antidote. Likewise, you might be too far gone. Are you reading what I say or have you instructed your agent to distill this essay into bullet points for easier digestion? Or are you perhaps listening to the voiceover of a generated transcription? Can you hear me at all—or are you already too braindrained?

Anyway, let me offer some ideas. A worldwide ban might work: let’s all deport these geniuses to the datacenter-shaped country they came out from. You can also move forward. Embrace “friction-maxxing,” a trend that travels parallel to “cognitive surrender” but in the opposite direction: let’s populate our lives with daily chores to forgo convenience and buildup tolerance for discomfort. Let’s pay homage to the domestic sublime to avoid sliding into death’s maw. Alternatively, if neither the collective nor the individualistic options suit you, we can take our chances with a dictatorship: let’s entrust politics to kindergarteners and set up an age of minority.

I’m afraid none will work.

A kid might not be strong enough to enforce the toughest policies; friction-maxxing is itself an indictement of its thesis, for it rests on itemizing serendipity; and, finally, the AI race prevents any attempt at global pacing. Besides, in case you haven’t noticed, those who could actually do something did not fall ill: how else do you explain that they still write their essays?

Substack tried to be a sane voice amid the anarchy of this quagmire. Partnering with an AI company to detect AI writing sounds like an oxymoron but a crazy world deserves crazy measures. A witch hunt, however, would only fuel a different kind of mindlessness: just like there are people who only have AI inside their skulls, there are people who only see AI outside of them. To be clear, I’m not sure I prefer to deal with a solipsist rather than a bunch of philosophical zombies.

I therefore accept we’re on the verge of a drastic class schism, a seasoning schism.

Those who find the optimal amount of each ingredient will rise above the tides. Those who fail to put some brain in their thinking will succumb, barely superior to the bots replacing them. And so the course of action I propose is to weaponize this phrase. Turn it into a slogan, into a battle cry.

I already have. Before, when people left AI comments on my writing, I used to “AI;DR” (it’s AI, so I don’t read) or insta-block them, but those are band-aids on a cancer. From now on, I will just comment below: “put some brain in your thinking.” I let them know that I can tell, as Sam Kriss would say. Don’t hesitate to spread the word.

In case you need convincing, let me tell you that, as an insult, it’s unmatched. If I call you a “stochastic parrot,” chances are you don’t know what stochastic is, or conversely, think it as a compliment. If I say you’re a “slop cannon,” you will say slop is in the eye of the beholder. If I say you’re “as conscious as Claude,” you will respond, proudly, that “Claude is as conscious as you.” But if I instead tell you to put some brain in your thinking, by the time you understand the inversion you will have already proved my point.

If you’ve reached this far I have good news. You have enough brain left to read above 12th grade and enough attention to read 2,000 words. And, more importantly, I’ve convinced you that your thinking is nevertheless missing some brain and you want to put it back. I’m glad you asked because I have exactly what you need.

First: take action.

Stop reading essays about how your brain is gone and delete your AI apps for a week. Let’s see whether you make it to next Wednesday. You think you’re faster at work, but are you better? Have you stopped to consider whether you’re more productive or just busier? Does your AI spending translate into outcomes or you’re just gambling at the token casino? Don’t ask ChatGPT to do your emails; don’t write code with Codex. Write an essay by hand. Read a book that’s not audio. Lift weights without using an industrial crane.

The second thing you will do, you won’t like. I’m only slightly disappointed that it took an OpenAI staffer to formulate the mechanism: “a lot of you would benefit from doing less and thinking more.”

In a world that over-indexes on high agency, you can get ahead by exercising your birthright as a member of the human species; make God proud by putting His gift to good use. Introspect a little. The reason why you scour the model market in search of replacement parts, is because you’re terrified of yourself. You’re terrified of realizing that, in a world that’s emptying itself of interesting things, you are not sufficiently interested yourself.

But trust me when I say this: you will be if only you put some brain in your thinking.

Anthropic has realized that if you promise the stars the Moon won’t suffice.

But let’s hear the numbers first.

$16+ billion revenue in a single quarter and 46% QoQ growth. Holy. Both figures are unprecedented for such a young company at this stage. Together, they might be unprecedented in history (maybe Apple). Not to mention the reported ~$74B in run-rate revenue. Anthropic is a beast of a business; not even OpenAI is that high.

What’s the problem then? Why are investors complaining about the GOAT? Because everything is unprecedented in AI, and so “unprecedented revenue” is not enough.

By the way, TickerTrends nailed it in the chart below. I did, too: “It only takes some finite amount of time to turn a prideful exponential into a disappointing sigmoid.”

What do you mean you are growing only 46% QoQ? It was 145% last quarter! What do you mean your exponential revenue has turned into a sigmoid? I don’t care if half the SP500 is losing value, you were the chosen one! When the entire American economy unambiguously rests on the tech industry which in turn rests in the AI industry, which in turn rests on Anthropic’s IPO being successful, any bad news is terrible news.

Anthropic is not allowed to slow down even a bit because, like Atlas, the world rests on its shoulders. Isn’t that what you wanted, Dario? Don’t you fail us now—where’s that country of geniuses in a datacenter, let me see it! In November 2025, I wrote that OpenAI was “too big to fail.” I meant that 1) the government wouldn’t let it fail but also that 2) were it to fail, everything would fail with it.

The same rationale applies to Anthropic now, which took the role of industry lynchpin from OpenAI this year. Tragically, it’s failing to rise to the occasion.

Naive extrapolations suggest Anthropic will fall short of its $100B run rate revenue target. Q4 must see an unlikely re-acceleration of revenue growth if they want to appease investors. Most likely, though, it will keep plateauing. I don’t think Fable 5.5—Anthropic’s ace up the sleeve, coming very soon—can move the needle; which use cases will it unlock that Opus 5.5 didn’t? That’s the key question: is the marginal gain from a new model enough to put the company back into exponential growth?

I just don’t see it.

Suddenly, the $2 trillion valuation that some of Anthropic’s backers are hoping for at IPO seems too much. As do the $518 billion in infrastructure commitments it has made for the next decade (80% non-cancelable). It also seems bad that the AI boom is the biggest in history, above railroads, dot-com, canals, etc.

Thankfully, as Dario Amodei said in a Summit in December, he is “pretty confident they’ll have the revenue at the right time.”

For Anthropic, exponential growth isn’t celebration but the bare minimum!

Am I perhaps being too demanding?

Listen, they said AI will soon change everything (recursive self-improvement by 2028—no, by 2027!), and so the market has updated accordingly. It will measure this unprecedented boom against an even more unprecedented set of expectations.

Anthropic’s revenue numbers are one sign among many that the bubble might be closer to popping than anyone thought. But there are many others.

Read more

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (back soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

“You know my method. It is founded upon the observation of trifles.”

— Sherlock Holmes

Related: The Actual Reason Why Google “Fell Out” of the AI Race Changes Everything

In early 2026, Meta and Anthropic became, briefly and improbably, very good friends.

Mark Zuckerberg has been, for the past two decades, Silicon Valley’s wunderkind. When it comes to AI, however, he’s made mistake after mistake, midwit-style. In 2014, he lost DeepMind because Demis Hassabis caught him bundling AI with peasant technologies (virtual reality), an unforgivable sin for a true believer. By 2015, when OpenAI was being born, he already had one of the best research teams in the world together with Google and a few top universities, and yet he missed out on LLMs. Meta quickly fell behind. But Zuck himself only falls forward, finding side doors in empty walls. He will do whatever it takes to win. He’s the human incarnation of undirected agency: wherever the wind blows, he’ll get there fastest. Now, after rearranging the guts of his company, he’s back at the top.

Dario Amodei, Anthropic CEO, is, in more regards than I can count, the extreme opposite of Zuck. A diehard idealist who thinks AI is conscious (even if he will proclaim doubt the moment you tell him that implies he’s a slaver). He’s the diehard-est believer in the power of AI, willing to make a fool of himself by fighting the Department of War to defend his principles; adamant Amodei as sharp contrast to fickle Zuck. To him, open-source models like Meta’s Llama family are a betrayal to America. To him, a personal superintelligence is an oxymoron, something only a fearful atheist will say. To him, Meta as a serious contender to the AI throne is, perhaps with the exception of China, the most likely existential risk to humanity.

And yet, the circumstances of the moment brought them together in April 2026. Like two same-side magnets, they were put and kept together by the inexorable grip of ambition.

At the time, Meta was barely alive in a race that saw OpenAI and Anthropic push forward even beyond Google. The humiliation was palpable: up to 60,000 Meta employees had access to Claude Code; engineers were using it all the time (they had to). The company was spending on the order of 60 trillion tokens a month (70+ trillion at the peak) between Claude and other models, and top users were gleefully tracked in an internal token-maxxing dashboard that were thoroughly mocked by the commentariat (they called this Claudeonomics), which prompted the company to shut it down. Why was Meta incentivizing its employees to give as much money as possible to Anthropic? No need for conspiracies on that one: Claude was just too good. OpenAI was busy pivoting to enterprise and had fell behind in agentic offering.

At one point, Meta projected that its Anthropic spending could reach $10 billion a year. It’s interesting, to be sure, that around that time Anthropic’s annual run rate revenue grew from $19B in March to ~$30B in April. In an article I was writing, I hinted at the possibility that Meta was singlehandedly propelling Anthropic into exponential growth. There was no reliable information then to back up this take. People implied I was crazy: Anthropic was okay.

Incidentally, as I was finishing the article, Meta launched Muse Spark, the first genuinely good model the company had released in years. Llama 4 had flopped; Avocado had been delayed. But in ~9 months, Meta put together a brand new team, fixed every mistake that had spoiled all previous releases, and successfully shipped a nearly state-of-the-art model: it scored 52 on the Artificial Analysis intelligence benchmark, toe to toe with Claude Opus 4.6.

I don’t believe in coincidences. And I bet neither Zuck nor Amodei do, either.

So, what actually happened there?

Read more

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (back soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

This is one of those times when I feel like I’m the only person who’s not blind.

Roon, an always optimist but also sensitive OpenAI staffer, urges us not to “be reduced to the status of audience at a magic show.”

Don’t stand there, he says, clapping your monkey hands. These mysteries are for you to grasp. Want to learn the magic trick? Want to take a peek at the show’s behind the scenes? Well, maybe grow 200 IQ points then.

He is, naturally, talking about OpenAI’s barrage of mathematical results.

It’s a document with over 300 math solutions—in varying stages of the verification process—many of them, if proven correct, of great historical importance; some of them even more than Navier-Stokes.

No, that’s still an understatement. Try this: If a human did something like this, they’d be ipso facto crowned “God of Numbers.”

That’s how big this is.

The conquest of math by AI has been going for months now but today is different. I’ve seen serious people say that, together, these results are the biggest thing ever to happen in mathematical history. That it’s the equivalent to a nuclear bomb for mathematicians. No hesitation or qualification whatsoever.

There are a bunch of Fields Medals hidden in the document. Or rather, the Fields Medal doesn’t make sense anymore because of that document. What sort of advance can render the main recognition of an entire discipline meaningless? This one.

But I’m not bitter. After all, I wouldn’t have been able to grasp these results had it been a human who solved them instead of a machine. To me, it amounts to the same, for I’ve been mere audience to a magic show all my life. It’s just that it feels weird that the magicians I used to look up to are now applauding next to me and it’s mathematics itself that’s doing the magic.

That is weird.

I said that roon is optimistic but he’s actually coping hard. He’s too smart—and too much of an insider—to start at the denial stage so he went straight to the bargaining stage. He says that the magic is for us to understand, but is it? Not only is the amount of stuff to go through insane or the speed at which it happened appropriately superhuman, but the complexity itself is bordering on the limits of what the best of us are capable of. Guys who spent decades in some of those problems are awestruck. Never before has the expression “being struck with awe” made so much sense.

And so I’m reminded, once more, of chess and Go.

Because whatever it is that AI conquers next, chess and Go already went through it: it’s “move 37” and whatnot all over again! And what was the conclusion then? That humans need not apply. We can’t keep up. The set of “best players of all time” couldn’t keep up. They admitedly learned quite a few tricks from the AIs, of course, and they even surpassed the limits we thought possible for an ape with an overgrown soup of neurons inside the skull. Amazing! But that’s it. Not only can Magnus Carlsen never ever win against Stockfish, he can’t even understand what it does or why.

It’s simple: learning from superhuman intelligence is not the same thing as remaining intellectually commensurate with it.

So why does roon, supposedly Bitter Lesson-pilled, imagine it will be different for math? Or physics or biology or whatever they decide to throw into the machine in a trade-off that can only be described as “progress despite us”? It won’t be different. We still see Carlsen’s magic shows because watching an ape do chess is fun. But math? Nah just show me those impressive results and I’ll applaud.

What roon and his colleagues in the AI industry are doing is, effectively, passing the baton to AI on behalf of humanity. We will no longer be proud cosmic wanderers but timid cosmic luggage—or worse, cosmic tourists. We are, certainly, no longer magicians at the show but awestruck audience at best.

I argued, when I first wrote about what would happen when they started indiscriminately pointing the AI gun at the math targets, that it would be a problem trying to make sense of all these incredible results. Insofar as we wanted to use them for what is, by all accounts, still a human world, we’d need a connective tissue. Someone who could receive the gift of magic and translate into material gains or knowledge. After all, the world can only make sense of a discovery if the intelligence of the receiver—not the discoverer—is enough to make sense of it.

Who will play the role of connective tissue, I asked. Who will go down into the machines’ dressing rooms to discover the meaning of their magic and then make it legible for us, inferior animals? Only a few can, I thought.

But naive as I was—or perhaps only hopeful—I made two serious mistakes.

First, thinking that there would always be someone who could. And second—even worse—assuming that such a role would continue to be necessary. I thought, and even said explicitly: the only scenario where it won’t be necessary is if this world ceased to be human. Of course, I reasoned, that will never happen. I realize I didn’t pursue that line further not because I believed it was impossible, but to protect my emotional well-being.

Well, it’s too late for that, roon.

I understand your intention—and praise it—as well as the psychological drive that makes you write that tweet. But you’re deeply wrong. You—and many like you—are directly responsible for the reality you’re now refusing to accept. Actually, I don’t know if you are coping or just gaslighting us. You seemingly want us humans—proud, adaptable humans—to maintain the subject-object relationship with the intellectual fields that the AI you make is destroying. It’s not destroying the disciplines themselves, to be sure, but the relationship we had with them. AI might kill us all, who knows, but now I wonder if it won’t even have the chance because we’ll all die first of sadness.

Forgive my pessimism here. I just can see into the future. I can see where this leads us. And I see we have no means of stopping it.

So dare follow the logic to its endpoint, roon, and dare face the consequences of your actions. There’s no hiding behind vain hopes anymore. Even you, at the epicenter of what is perhaps the most important transformation this world has experienced, can’t accept it. You’re an actor in the last human show, but I don’t feel like applauding.

I guess it’s true that “courage is in shorter supply than genius.” There isn’t any more courage in trying to reach the stars than in admitting they are, perhaps, too distant.

And yet, as you build a stairway to the stars with this magic of yours, you encourage the human to jump, like the ape he is, to reach just as far. If he only jumped high enough, you say, he may brush the rim of the firmament. Just a bit higher… And in doing so you pretend this truth doesn’t need reconciliation: in making this world more magical—what a time to be alive!—you’re also making it less human.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (back soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Anthropic might be screwed.

The AI industry loves to defy nature—like when they go and try to build an artificial God—and they often get their way—like when they convinced the US government to splatter America with a bunch of ugly, noisy buildings in what were otherwise beautiful landscapes—but there’s one battle the industry will always, inevitably, lose: nature can’t stand exponentials.

This is a fact of life: I grew twice my size in the first year after being born and then doubled again by 6 or 7 and now I’m biologically stuck at ~180 cm. But I’m Spanish and I know for a fact that things work differently in California. The Silicon Valley hivemind—I’m not talking about agents—has developed a mild exponentiality disorder: the number of transistors inside chips, GDP, AI benchmark scores, the size of the models, intelligence, and the amount of attention they can steal from us before we rise up in arms—all those things are expected to point northeast indefinitely.

Exponentials are, to them, like morning caffeine and weird parties—a constant.

So far, they seem to be correct. Nature, however, does not tolerate insurrection (that’s why the sun sets in San Francisco).

An exponential can last for a while—see, COVID—but it always, eventually, finds a binding constraint. In the case of COVID, there are only so many people a virus can infect (see also, Pluribus). For human population, we used to think food was the limiting factor but turns out screens are. In the extreme scenario—beyond Bay Area delirium and microorganisms—one always encounters a finite observable universe.

The cosmos imposes a rather annoying upper bound on how fast things can grow.

To give you a sense of how rapidly exponentials exhaust, take a chessboard. If you put one grain of sand in the corner and then double the number for each consecutive square, by the time you reach the opposite corner, you have stripped every beach on Earth of sand for your stupid little game.

But then, with all the sand gone, how will the industry manufacture any more silicon wafers for their chips?

Wrong question! Some exponentials manage to avoid their fate for longer than usual. It was Gordon Moore, Intel co-founder—although probably better known for the law named after him—who said that although “no exponential is forever,” forever can be delayed.

Moore’s law has lately found itself dealing with quantum tunneling and all sorts of magical properties of atom-sized particles but before that, it had an impressive run of five decades of doubling the number of transistors in a chip every ~two years.

The problem is that it only takes some finite amount of time to turn a prideful exponential into a disappointing sigmoid. Here’s what a sigmoid looks like, for illustration purposes (courtesy of mathematics):

A plateau is better than a downward slope, sure, but it’s not sufficient for the singularity. This is the main mathematical objection against the naive version of the AI industry’s preferred thesis: every vertical line eventually discovers it’s just early.

“Wait,” says the AI industry learning of this unwarranted pessimism, “what if we stack a bunch of exponentials-turned-sigmoids on top of one another? Instead of adding increasingly more transistors to a chip why don’t we also increase, say, their individual performance! And the size! That would transform a bunch of S-shaped curves put together into one big exponential again!” They are correct:

So clever.

A sigmoid that begins when another ends creates, if you zoom out, the perception of true exponentiality. Silicon Valley has used this argument for years to dismiss mathematical skepticism. In theory, it works: technology doesn’t grow one thing at a time but different variables influence one another, creating much more powerful growth mechanisms. The population explosion, for instance, wasn’t caused by exponentially more food alone, but by mutually reinforcing improvements in yields, sanitation, medicine, and energy availability. The same applies to chips and AI models.

Unfortunately, the exponentialists fail to realize that a stack of sigmoids on top of one another, insofar as it can truly resemble an exponential, also obeys the laws of physics and is in turn doomed to also become, ultimately, a sigmoid. Unless you have an effectively inexhaustible supply of new S-curves—you don’t—your efforts to deceive nature are in vain. Sigmoids can be delayed for a while but, like the grim reaper, Gaia eventually catches up.

This might be a contentious assertion so I will say, admitedly, that stacking S-curves can approximate an exponential for a long time, enough to outlast humanity (in that case, the binding constraint would be, sadly, our extinction). But that’s tangential to my point, which is this: Silicon Valley’s love of the indefinite exponential growth is contra naturam.

My point is not that all exponentials die fast but that they eventually die. Period. And so we shouldn’t be surprised when it happens.

It follows, naturally, that not all exponentials fold to reality equally easily. Some are stubborn and turn into sigmoids slowly, like Moore predicted of chips, others survive by escaping into subsequent S-curves, like AI model performance (size, data quality, better chips, inference-time compute, etc. all collaborate). And yet others quickly fall before the laws of common sense due to factors as mundane as a presumptuous forecast.

Having now definitively settled a debate that has occupied computer scientists and philosophers for decades, we can move on.

Apropos of nothing, let’s take a look at Anthropic’s revenue growth.

Here it is with the trendline drawn on top:

Uh oh!

I could end this article here. My job is done. Sell the S&P 500 index fund in your 401(k) and go home with your kids. I’m kidding, I would never give financial advice when the bubble is about to pop. Only God knows what comes next, although you can try asking Claude.

What I will do, however, is examine the possible explanations to a simpler question:

What the hell is happening to Anthropic?

Read more

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (back soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Last week I published an optimistic article about a post-AI world. Today’s essay comes from a more pensive and somber place. How to deal with what’s coming is the most important question right now, and it deserves a full treatment: the light and the shadow.

I.

I am standing on the fifth-floor balcony of my hotel room in Lucerne. It is as beautiful as it is old, two traits buildings tend to accumulate together until the latter kills the former. Before me, the Lake of the Four Forest Cantons; large, but not so large as to conceal from me the lush sweep of its quiet shores. Below: swans, coots, mallards, and cormorants swim blithely. In the distance, the Alps.

I look at their majestic stillness, peaks posing for an eternal-exposure photograph. They are unmoved by the scarce clouds snagging on their serrated ridgeline, by the wind waving green with all its trees, by the dance of a timid sun and a crescent moon. They are a snapshot of the past. And of the future, too. A million years ago they stood there and in a million years—when the Montana hotel will have long crumbled down the hill into the ruins of Lucerne, the lake will have dried out into a rocky bed, and the conifers and waterfowl will have turned into some strange new forms of life—they will stand there, just as impassive.

Lately, I feel mountains are among the very few things of which I can say that.

The world is changing. So said everyone across the ages; it always has, of course. Don Quixote thought himself born “by Heaven’s will, into this our age of iron, to restore within it the age of gold.” Ovid, more cheerfully, thanked his luck: “Let others praise the olden days; I am glad I was born in this present age.” Nostalgia or enthusiasm, the premise is the same: one age gives way to another. But the size of the clock is different now. Historical change used to be measured in millennia, perhaps centuries, sometimes generations. I am talking about years. I am talking about months. It is “weeks where decades happen” all the way down now.

The world is changing faster than any person can absorb the change; we can barely pay the interest to a growing debt of existential proportions. Planet Earth was never built to metabolize humanity at this scale—see the Industrial Revolution and its consequences—and likewise the human animal was never built to metabolize technology at this speed.

I go for a walk through the city with my girlfriend to welcome the arrival of autumn, already visible in the orange, brown, ochre, and yellow of the fallen leaves and in the fact that I’m not wearing shorts. The cosmos keeps to its usual rhythm, but I find myself amid the unfolding of something extraordinary. I see the people passing by, smiles on their faces as they leave the old town. I take my phone out to take a picture but open Twitter instead due to years of conditioning. My timeline is a jumble of calls for governments to take AI safety seriously, employees at the labs warning that we have no idea what’s coming, and model after model after model keep coming out, as if the whole thing were a Victoria’s Secret runway. I snap back into the tranquil cityscape and the smiles are still there. What. My screen is melting away into chaos and you are smiling? I feel a sudden urge to scream.

I contain myself and put the phone back in my pocket. With my hand still wrapped around the metal slab, a sudden sensation takes hold of me: I’ve been caught at the edge of a black hole. One hand already past the event horizon, stretched thin into the dark; the other still here in Lucerne, with the impossibly high Alps as a backdrop, waving summer away.

II.

People call this frantic pace of change the Singularity or the exponential or a takeoff or a Cambrian explosion. I do not particularly care what it is called. Nor, for the purposes of this essay, do I particularly care whether the cause is a successful Bay Area cult leader or some primordial force such as gravity or capitalism.

I am more interested in what it is like to be a powerless person carried along by it.

I say it hypothetically—like one asks “what is it like to be a bat,” for instance—because the changes do not affect me. I write about these events with the dispassionate detachment more befitting a chronicler hovering over the world (or a guy on vacation) than a peasant about to be swept away by the tide. How could it be otherwise? AI mastered software but I have never coded professionally. It took over the white-collar economy but I am not a designer, nor do I work in the corporate world. Even mathematics, for which I have a more particular affection, lies far outside my own preoccupations. The other day I wrote, with vicarious astonishment, that OpenAI solved Navier–Stokes but I did not feel it under my skin because today I can write about the Alps and I rest assured they will be there for me.

Insofar as I present myself as respectful of the grief of others—soon to be the majority—perhaps that empathy has been cheap: I can afford to spend my time thinking about them at all. I even allow myself a certain smugness because I spend my afternoons and sleepless hours in a discipline that seems to resist AI. Taste cannot be verified or trained against, I say to myself. Claude was given a human name, but it wasn’t given Nabokov. And though my work is, nominatively, an “email job,” the parasocial relationship you guys have with me is irreplaceable. You’re the Alive Internet Theory.

But I know exactly the trick I’m playing. I’m professing safety on the grounds of ignorance. It’s from not knowing that I sleep full nights. It’s from obliviousness that I can walk around Lucerne as if I had nothing else better to do like, say, invest in an offline bunker. Am I really beyond all this trouble? I still have room for preparation, sure, but no longer for skepticism.

Others who, like me, have skeptically seen AI take over one thing after another except their own, have finally come to terms with the truth.

Scott Aaronson writes in a recent essay that, if we are “intellectually honest,” there’s no denying that we’ve crossed some significant line sometime in the past year: “you’re no longer being asked to believe in arguments and extrapolations, but only in the front-page news,” he writes. He’s correct; GPT-6 Astra is too good. Opus 5.5 is even better. GPT-6.1 will be better still. And so on, in perpetuity. Insofar as you want to remain a useful skeptic, you have to work harder to deny the reality being published every week than they have to work to solve the next Millennium Problem. You may use the help of a chatbot to beat the future shock of needing a chatbot to help you.

Aaronson makes an important qualification, though. “The singularity has already started,” he writes, but it’s “wildly unevenly distributed.”

III.

I’ve made that point before.

AI is a strange technology for one reason above all others: its general-purpose nature means that its capabilities are distributed unevenly along many dimensions at once. In other words, AI can feel like a nothingburger to your mom and like the end of history to you.

Here are some specifics. Access is uneven. Model quality is uneven. Money and compute are uneven. So are technical knowledge, frequency of use, willingness to delegate, familiarity with the domain, and confidence in what the technology can do. The result is that someone at one end of this distribution—perhaps your mom, occasionally dipping in short chats with the free models, displaying little savviness or trust—inhabits a radically different reality from someone at the other: you, with deep knowledge of both AI and the field of application, constantly deploying swarms of agents on an on-premises server.

This is the opposite phenomenon I alluded to in the beginning: the Alps are, across many times, one; we are, within one time, many.

It is is the familiar idea that “the future is already here, just not evenly distributed,” except that AI multiplies the effect. When Aaronson says that the singularity itself is unevenly distributed, he is pointing to the fundamental axis: tractability. The labs have mastered the elusive art of raising the ceiling of AI capabilities but can’t seem to learn the basic practice of raising the floor. You have an Einstein in your hands one moment and a random two-year-old the next. You have fully saturated benchmarks but 2% GDP growth. Etc. Etc. The shape of AI is a spiky star.

But it might not always be like that.

That’s what we have to come to terms with. The singularity might turn AI’s shape into a perfect circle, engulfing all domains as if they were all equally easy. As if Moravec’s Paradox was a temporary artifact of anthropocentrism. As if Einstein being two or fifty-five made no difference at all. As if judging a monkey or a fish for its ability to climb a tree was just as damning. As if William Shakespeare and Alberto Romero were indistinct names.

Could the future be forever as jagged as AI models are today? Sure! But although I have the hope, I don’t have the certainty.

Everything **around me—the lake, the angry swans, the Church of St. Leodegar, the Alps, the moon, my hands—looks exactly like it did in 2000. Will it tomorrow? “The ultimate test is reality,” I wrote, but what if reality folds?

It’s in this sense that the wall behind which I take shelter is precarious. I’m waiting for a siege fierce enough to break it, with a terror so deep it cannot express itself as terror but as cope. To the extent that the coping mechanism is working, I sure praise the wall: it’s made of non-verifiable domains, it’s made of a real world that’s tougher than paper tests wilting in the mildest storm, it’s made of superhuman AI falling short of omnipotence, it’s made of the Alps standing immutable for a million more years. It’s made, ultimately, of meat. But what if I’m wrong. What if the wall is brittle, short-lived, not tall enough, made of granite, scalable.

I’m terrified that I’m wrong. Scott Aaronson, former skeptic, is likewise terrified:

My position on AI is merely the conservative, skeptical position of 2006, updated with intellectual honesty for the reality of late 2026. And that position, if you need me to spell it out, is as follows:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

IV.

As I scream alongside him, I lift my head from his essay and the indifferent moment hits me in the face. I somehow arrived at a café on the banks of the Reuss. My girlfriend is sitting in front of me. She smiles at me. What are you reading, she asks, you seem tense. She doesn’t know the world is ending. I smile back. Look at the river—it’s just so pretty, I say. Everything looks abnormally beautiful; there’s a sort of domestic sublime to abandoned boats and floating debris when the end is nigh.

The Chapel Bridge is to our left; to our right, free-tour crowds wander past, literally and metaphorically. A woman has just left the table beside us. She smoked an entire pack of cigarettes in ten minutes. Sensible woman, I think. Perhaps her body knows she will not have many more opportunities. Perhaps it’s the exact opposite: she knows smoking is the only thing left for her to do. In any case, she gets up, and a lame pigeon takes her place.

One of its legs is rotting. Every few steps it lowers the ruined foot to the ground, just long enough to catch its balance, then carries on pecking at the breadcrumbs left on the plate. I watch it for a while and can think of little else. One good leg, the other nearly gangrenous, something close to a death sentence written onto its body, and still it pecks away without pause or restraint. A flock of sparrows awaits at a distance. I imagine they are respecting her fierce hunger, or are perhaps aware of her imminent demise, and are instead respecting her last supper. It’s a brief yet profoundly courageous existence, that of the pigeon; visibly failing and yet not a second surrendered to what has already been decided for her.

I look at her and feel an eerie recognition. The pigeon is me. The pigeon is all of you, minding your own business—pecking away at the crumbs you have left—with one fewer hand and pretending nothing ever happens. Don’t get me wrong. We are not pigeons in the literal sense (although I have quite a sizeable audience now so one can never be sure). It’s just that we might as well learn something from this pigeon in particular. Acceptance maybe, as Aaronson seems to suggest? Or is it resilience instead that I hear in its steady pecking: here, here, here, here, here.

Morsels are on the menu today and forever so you might want to adapt your appetite; just like the one-legged pigeon lives in a frail relationship with necessity, now we do, too.

Annie Dillard—RIP; thankfully she didn’t have to witness an actually terrifying total eclipse—writes that “it would be well, and proper, and obedient, and pure, to grasp your one necessity and not let it go, to dangle from it limp wherever it takes you.” In our times more than in hers, that means to fully indulge in what you love and not look back, for the silicon invaders intend to erase the already feeble importance of all considerations other than love.

There are no interesting things, I wrote in September 2026, only interested people.

We will always have the kisses, I wrote in July 2024. We will only have the kisses.

V.

It is from a position of love that Erik Hoel—who, being a writer like me and a scientist like Aaronson, has remained skeptical due to AI’s suspicious inability to write good prose—says we should not surrender to an unwritten fate.

In March, Hoel wrote that “it’s been six years since GPT-3, and there has been no ‘move 37’ moment for writing.” Move 37 refers to a move played by DeepMind’s AlphaGo against Lee Sedol in 2016. AlphaGo would go on to win with a devastating 4–1. The game became an inflection point in the imagination of the AI world: a machine had “produced” a play that expert commentators initially found baffling and later described as creative and ingenious. Go had long been considered a much harder target for AI than chess, largely because of its vast search space; could AI do the equivalent in writing and outperform the best men of letters today? Not by a long shot.

Other writers have echoed the sentiment, even those who take AI seriously. Jasmine Sun says the unlived life is not worth examining: AI “cannot live, cannot feel, cannot smell, cannot taste, cannot sense” and so it can’t have an authorial voice. Adam Mastroianni says AI “can only read the cookbook; it can’t taste the meal,” or that its writing is “all necklace, no neck.” Sam Kriss writes that, “However bad a writer you think you are, you are not worse than AI.”

I agree. I believe Hoel does, too. But he, like Aaronson, just published his attempt to reckon with the reality I speak of here. He writes:

Maybe I’ll be back to the more skeptical side in the future: e.g., math and programming are both verifiable domains, and it’s suspicious that the first blatantly superhuman AI progress is in those. Maybe aesthetic taste and research taste, truly novel ideas, and continual learning, are all difficult-to-replicate hallmarks of human consciousness. But am I 100% certain? How certain could anyone be? Are you sure OpenAI won’t set aside a hundred million dollars in compute, and have their swarm of a million agents think for the internal equivalent of a million years, and at the end it spits out a way to better render domains like “taste” verifiable?

I suspect he’s speaking out of respect toward the unknown rather than deference toward the industry’s forecasts. After this belief update, Hoel flew to Washington to support a ban on superintelligence. His point is clear: death before disempowerment. We are better as ashes in a pot than as the pets of a bot. He won’t help hyperstition the industry’s preferred future into existence by treating it as inevitable.

VI.

I stand with Hoel.

I find both his reluctance to accept that the ineffability of taste can be reduced to binary—we’ll see about that—and his defiance of the machine appealing: “In a future with actual superintelligence, brilliant young people wouldn’t be the ones to start companies, or make discoveries, or decide the fate of their world. It would all be done by humming banks of machines out of sight with immense influence.” Let’s keep humanity in-the-loop! Otherwise, this prediction I made in March of 2023—more with analytical neutrality than chauvinistic loyalty—will become real:

Computers will keep running toward the unknown long after we won’t be able to recognize our fate anymore . . . soon, we’ll be just spectators, mere observers of a world neither built by us nor understood by us. A world that unfolds before our eyes—too fast to keep up, and too complex to make sense. . . . It was bitter to accept that, after all, we might not be the key piece of this puzzle we were put in. It’ll be bitterer to finally realize that we’re not even worthy enough to partake as sense-makers.

A perfect game of Go is unintelligible to any human and humans just go about playing Go as if nothing happened. So why should a discovery or a company be any different? Simple: a game of Go, however immaculate, does not aim to build reality on your behalf.

Some argue that this is a ridiculous prospect. Daniel Litt, mathematician, says AI will neither “make us stupid” nor “obsolete us.” So did David Hilbert, mathematician: “We must not believe those, who today, with philosophical bearing and deliberative tone, prophesy the fall of culture and accept the ignorabimus. For us there is no ignorabimus, and in my opinion none whatever in natural science. In opposition to the foolish ignorabimus our slogan shall be Wir müssen wissen—wir werden wissen (’We must know—we will know’).” Hilbert was well-intentioned but ultimately wrong. Terence Tao, mathematician, asks: “What if an AI tool generates a lengthy proof that nobody—not even the original humans prompting the AI—understands?” Tao asked that on July 26. On September 8, OpenAI made it real:

[M]athematicians say that, so far, humanity has learned very little from the solution [to Navier-Stokes] the company’s AI model purportedly found. Although they believe the proof to be technically correct, the dense, 166-page manuscript drafted by AI is proving to be a difficult read.

AI can kill the bottleneck of human stupidity, but in the process, also erase our role in this cool story: once travellers of the cosmos we’ll be relegated to cosmic luggage. Aaronson wonders what his kids will do when they grow up—his 13-year-old daughter said, rather hilariously, that if “she wants to become a mathematician, it now looks like she has maybe two more weeks”—but he nonetheless advocates for acceptance. Hoel, for resistance.

Shengyu Liu exists in between.

VII.

Liu differs from both Hoel and Aaronson in the sentimentality of his posture, the country of origin (China—uh oh), the fact that he was never a skeptic, and, crucially, in how far he’s gone into the lion’s den: he’s an industry insider.

As a kernel engineer at DeepSeek—if I’m correctly interpreting his role and measuring the importance of his contributions to this freak show—he’s one of the main guys responsible for AI getting so good. His work involves tinkering inside GPUs to make the models be so efficient, so cheap, and so fast, that OpenAI and Anthropic are genuinely terrified China might catch up. So much so that Dario Amodei is willing to include China by name in his proposal for a global pacing effort.

Liu wrote an essay on how AI has affected his job and himself personally. Here’s a relevant excerpt (translated by Teortaxes and Astra and reformated by me for clarity):

When everyone is this determined to engineer their own obsolescence, I have little choice but to join this brutal arms race. . . . When the day really comes that AI is better at writing kernels than I am, what will happen to me then? My own judgment is this: I probably will not lose my job, but I will have to change what I do. I should still be able to make a living. But I may no longer have the chance to do the work I once loved.

. . . there is something genuinely painful about having the thing you love taken away from you. That quiet contentment of sitting at my workstation, settling in, and spending an entire afternoon writing kernels may sing its swan song this summer. I have no choice but to bury my talent in yesterday and become a mech pilot. There are more gears in my hands now, but fewer rhythms in my heart.

Love is a powerful force, sure, but perhaps not one available to everyone in a post-AI world.

Like Annie Dillard, Isaac Asimov told us to do “whatever we wish,” but unlike her, he accurately factored in machines’ conquest of human pursuits. We could become aristocrats. Life would not lose its meaning, because the idea that work is what gives life meaning is relatively modern and, in some sense, antithetical to human nature: we’d would rather play and gossip. This is what Alex Imas and Clara Collier, among others, argue. “After work,” Collier says, “we’ll have each another.” How wonderful. The real treasure really is the kisses we give along the way.

But it does not always seem to be enough. Liu does not want to be an aristocrat. He wants to turn gears. And though he now has more in his hands, his heart mourns the lack of rhythm.

VIII.

To Liu—and Aaronson, Hoel, me, artists and mathematicians and many others—life and work are two sides of the same coin. It is easy to say “do it as a hobby and stop whining!” but AI is forcing him to split up his life.

He had found a way to combine into one the thing he’s good at, the thing he’s paid for and the thing he loves. Isn’t that everyone’s goal? Isn’t that the life we all would like to have? Isn’t that the life we are desperately trying to achieve? Insofar as you’ve given up it is because you find it unattainable, not because you wouldn’t like to switch places with him. People who don’t love what they do will never understand the size of his tragedy: yes, he can do what he loves on the side, but he will hate his job and won’t have as much time for his lover.

Just like most people, right? That’s what Liu grieves, not the loss of his ability to tinker with kernels. He grieves having been expelled from an earthly paradise. He committed the unforgivable sin of finding a way to enjoy an existence that’s otherwise quite crude. Thankfully, those who, like him, were having a good time in this our only life are now as miserable as the rest of us.

He also grieves the loss of thymos in Fukuyama’s sense: he won’t be able to channel a subpar tinkering into a larger purpose, much less be recognized for it.

At first glance, this anxiety makes little sense from a historical perspective. Did progress not free us from the fields and the factories? There was surely some skilled people there. The best plowman in 10th-century England is no more an argument against modern agriculture than Liu is against whatever AI does next. So why not let progress free us from this too? From GPU kernels. And from math. From everything. I get the idea: progress has undoubtedly improved human life, and since we have no reliable criterion for knowing when we have reached the “optimum”—the point of balance between having a life worth enjoying and actually enjoying life—perhaps the best course is simply to keep going.

But taken to its limit, the logic breaks down. It is like reasoning that because a little water cleans a wound, we ought to drown ourselves in the lake. It is failing to realize that if you start sanding down a mountain’s rough edges you will have a nice hill at first but if you never stop, eventually you’ll find yourself facing a plain. Not knowing where to stop does not mean there is no point beyond which we have gone too far. A “push-button marvel,” as Noah Smith puts it, sounds wonderful. But Smith doesn’t realize that push-button marvels are an oxymoron. As I wrote elsewhere, “Just like heroism is self-destroying—it needs a dying world to save—the ‘modern world of push-button marvels,’ which accurately describes what happens when you prompt ChatGPT with ‘do a breakthrough’ and it does, is self-perpetuating.”

To be clear, my position here is not “stop today.” Nor tomorrow. Nor should we have stopped last year. Nor after cancer is cured. Nor before that, surely. The problem with drawing any arbitrary line is that, once you draw it, it becomes impossible to feel entirely satisfied with its placement. My point is simply that never stopping is just as bad as stopping too soon. I don’t claim to have an answer here. I do not even know whether this dilemma has an answer at all, but I do not think ignoring it is one.

IX.

Perhaps my aspirations are hopelessly romantic; perhaps it isn’t warranted for an overgrown ape with a propensity to enjoy its own reflection in dirty puddles to want to be as eternal as the Alps. With them as witnesses, I am listening to the swan song of the birds swimming oblivious in the Reuss. Summer is over. For the first time in my life, I am grateful to be finite. Alongside Liu, the swans, and Lucerne, I weep.

It has started to rain. It pours, in fact.

Where only a moment ago the warm September sun shone, clouds fully cover the sky and heavy raindrops stipple the river’s skin. It is one of those sudden showers that arrives without warning. No starting gun or pleasant introduction from any of the organizers. No welcome and no permission. Exactly what happened with AI in 2022. The storm is, essentially, an omen or maybe an echo; if I believed in the cloud-gatherer, I would no doubt be able to read its message all over the formless gray sky.

I open the umbrella that I had the sense to bring with me this morning and realize that nobody else seems to have one. They were not expecting the tempest. I see the startled crowds take shelter beneath the great wooden bridge. There are too many people. The rain is getting heavier. How many can fit? How much can the bridge withstand? Do they think a few planks thrown together can resist the wrath of a god?

I’m afraid they will soon learn the answer. I, in turn, retreat behind my wall up the hill. Back at the hotel, I sit on the balcony. The view is captivating: the mist has swallowed the lake, the valley, the entire city. Even the Alps, normally magnificent against the sky, have been reduced to the faintest outline. It seems that, in the end, not even the mountains will remain. It is appropriate, I suspect, that today is the last day of my trip.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid sub, here’s a button for that:

Subscribe now

An unorthodox guide for a post-AI world.

Half my social media timeline is “you are going to be obsolete any day now” and half is “you are going to be dead any day now.” Not sure which fate I prefer: to see the rock roll over my limp body or to watch it roll up the hill by itself?

But I do know something.

I’m not alone in wondering if there is anything I can do to secure a reasonably tranquil rest of my life (whether that is another forty years or four weeks). Below are some candid thoughts neither ordered nor fully coherent.

First thing is to stop reading people whose whole identity is thinking about 2030.

Those who want the thing “the sooner the better” or those who’d rather kill themselves are both bad guides. Whether they are right or not—irrelevant—they can’t help you. Their currency is your anxiety. They think they’re helping, but of what help is it to tell an ant that a boot is about to crush it against the ground?

If the future is jobless, you don’t want to strain yourself for that; if it’s unpredictable, you don’t want to think about that; if it’s a human genocide, you don’t want to partake in that. Start searching for your own meaning or come laugh at the meaninglessness of existence. Your call.

History can scream; you have a life to live.

“Plan for AGI” is not a plan. How do you prepare? Learn Python? Neuralese perhaps? Buy land? Where. Have children quickly? (Don’t have children at all?) How could I bring a new life into a world where it won’t even have the opportunity to become the best chess player in history? That’s immoral. Become a plumber? I’ve watched Manchester by the Sea. Become a GPU? I’ve watched The Matrix.

The best standard advice eventually degenerates into the habits of your average conscientious middle-class adult: save money, stay healthy, be kind to your neighbors, don’t become useless meat without the internet (they are so with the internet), find something you love and pursue it obsessively. Audendo virtus crescit, tardando timor. Etc. Etc.

You can also hit the barbells: either go all in on AI or assume that “nothing ever happens.” Or do them both in alternate weeks—they are good advice.

To be honest, you should not need this post at all, but I have the suspicion that you guys are solving the wrong problem. You keep asking how to remain “economically valuable” in a world soon-to-be populated with superhuman machines. Why? You will not care about money, either because it will be worth nothing or because the elites have figured out how to redistribute the robot tax to the underclass.

There is no in-between because no system, however robust it might appear, survives having its entire population lose its stake in it.

But, but the transition…

There is no transition—this is the transition!

Insofar as the transition could be prolonged by geopolitical or economic factors, the answer is what I already said above: the less time you spend listening to AI gurus, the better prepared you are to deal with a delayed takeoff because those quirky cult-prone dudes are often nerds, autists and overall people with poor people skills. The last remnants of humanity to fall to the machine will be those that resemble a high-school lunch break or, conversely, 18th-century Versailles, both of which are absolute hell on Earth for the computer types.

What you actually need—whatever happens—is to remain interested.

This is more difficult. With or without AI, it is difficult. A machine can take your job tomorrow and you will rebuild yourself somewhere else; boredom is forever. People often say you need to find your own meaning—e.g., me, above, ten paragraphs ago—but this is a category error. Meaning is an object in search of a subject. You.

There are no interesting things; there are only interested people.

You found all your meaning at work? (Doubt it.) Ok, then you will have to revisit your self-image. As I see it, there are worse outcomes than being obsolete. I know several obsolete technologies I’m extremely fond of. Fire. Parchment. Crossbows. Math (ouch). As a wise anonymous random sage once said, “there’s a perfect correlation between people worried about loss of meaning from automation and people who never cook for themselves.” Yeah: pick a pot.

What about you?

Me? I want a house. A big cottage in the Swiss countryside if possible, but failing that, I’ll settle for any house anywhere. This desire somehow survives every version of the future I can imagine. AGI: house. No AGI: house. AI winter: house. Ten years of 25% GDP: somehow, still, house. Kardashev level three? You guessed it: a galactic-themed house somewhere beyond the Andromedan plains.

My problems are pretty mundane, you see, so maybe my solutions are, too. A related thought: perhaps I should own more things that exist in Euclidean space. You know who will not be made obsolete by AI? Stamp collectors. No further comment.

I used to think that the sensible reaction to AI was to become better at the things AI could not do. That lasted four years after my birth in 1993. This now strikes me as an exhausting way to live. Every six months the fence moves. Coding? Gone. Mathemathing? Gone-ish. Writing? (Never, of course.) The point is: I don’t want my hobbies chosen by binary classifiers.

I can do whatever I wish.

What should get through to that thick non-silicon skull of yours is that there’s no “better vs. worse” dimension in the future. Hierarchy is a non-concept among pets. Humans defining ourselves by that was the mistake we made all along. What does a better “chill afternoon with the pals” even mean? It’s when you don’t obsess over how good the kiss is that you really learn how to give a good one.

So the obvious hedge is, again, other people.

This solution sounds suspiciously wholesome for a story about death and obsolescence and the end of purpose, so let me make it less so: machines will write better than I do, even if it looks less likely by the day, BUT my girlfriend is unlikely to replace me because Claude suddenly knows how to crack a deadpan joke. The local market for being Alberto remains pleasingly uncompetitive.

It’s in this sense that I advise: the only jobs that are intrinsically safe are those where the relationship between the people involved is the most valuable aspect. Nurture your tribe. You and me. Here. We are irreplaceable, my friend. People ask me all the time why I’m not scared of AI and the reason is that there is nothing AI can do to replace our memories together.

What makes you you? That.

So be interested.

And be interesting.

There is also the possibility that none of this happens at all.

I’m only writing this because my timeline won’t shut the hell up. But they may as well be wrong and Gary Marcus be vindicated. I would find it extremely funny for those thirty-something millennials to reach retirement age having spent thirty years emotionally preparing for impending posthumanity.

“Dad, what did you do in your thirties?”

“Mostly refreshed ai-2027.”

As advice for the younger readers out there, I return to manual competence precisely because I have none. Cooking (well). Fixing things or making them from scratch. Knowing what the fuse box does; knowing what a fuse box is. Recognizing a tree or witnessing it paint the day green with all its leaves. Plumbing is actually fine.

These are not AI-proof skills in any meaningful sense but I think you could use fewer layers between action and consequence. In this sense a post-AI life rhymes with a pre-technology life. We’ve come full circle. We’ve spent some 10,000 years adding layers between ourselves and the world and AI will cut right through all of them.

But if you remain unconvinced and don’t want to think much about these advices, the best argument I have for you is plainly empirical: look around.

Today is made of carbonated beverages, carbon-eating plants, carbohydrates and hydrocarbons, and a favorite between the ages of 0 and 33: carbonara spaghetti. Just like the world of yesterday. Meanwhile, the singularity somehow remains a perpetually imminent eschaton. So, to hell with silicon, it has a long way to go.

I don’t know what survives AGI. But I know today I don’t really care. Tomorrow-me asks the same question. If nothing relevant has changed—and given what I’ve said, I doubt it—tomorrow-me reaches the same answer. Then the same is true of the next tomorrow. I call this recursive self-contentment.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid sub, here’s a button for that:

Subscribe now

I’m back from Switzerland! I have a ton of things to tell you and a pretty considerable backlog of articles ready to go. I’ll get back to the weekly guides and reviews as I find the time (I’m covering pretty much everything in the weekly essays). Today, I bring you part two of one of my favorite formats: 11 charts that show the side of the AI boom the industry would rather you didn’t know.

In May, I wrote a chart compilation piece to capture the less press-friendly side of the AI industry. I’ve done it again with less-known charts and updated data.

For this second part, I’ve compiled another 11 charts focusing on the financial and economic side of AI. Together, they reveal the shape of a potential bubble. Rumors about the AI bubble peaked in late 2025 and then waned, but they’ve been gaining traction again lately.

These charts are a good way to see that results on paper (e.g., how good GPT-6 is on benchmarks) do not necessarily translate into results in the real world, at least not right away. And that an industry can fall amid its greatest successes. For all the mainstream alarm over the chance that AI might end up killing us, the ultimate test is reality, and AI is still failing it.

The format is the same as in part one, intended for a quick digestion: title, chart, one-liner explanation with source. In 5 minutes, you’ll learn more about the state of AI than you would in months of industry-grade marketing.

Here’s the outline:

1. THE TOP 10 COMPANIES ARE 40% OF THE S&P 500

2. DATACENTER CANCELLATIONS HIT A RECORD

3. INVESTOR LEVERAGE IS AT A HISTORIC EXTREME

4. MARKET CAP IS UP FOR AI COMPANIES BUT NOT MUCH ELSE

5. THE VALUATION AS A MULTIPLE OF REVENUE OF AI LABS IS BIG

6. HEDGE FUNDS ARE SELLING SEMICONDUCTOR STOCKS

7. THE AI BOOM COMPARED TO PAST BOOM-BUST CYCLES

8. STRONG DECLINE OF AI HYPERSCALER FREE CASH FLOW

9. AI SUBSCRIPTIONS CAN BE HEAVILY SUBSIDIZED FOR POWER USERS

10. HYPERSCALER CAPEX TO SURPASS $1 TRILLION BY 2027

11. OUTPUT IS NOT OUTCOME: TOKENS VS. LABOR PRODUCTIVITY

Read more

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Dario Amodei, CEO of Anthropic, published an essay called “We Must Pace the Frontier” that could define the next years of the industry.

He proposes that the industry slow down capability improvements while it figures out a safe way forward. His argument is that AI is advancing too quickly—with the help of AI itself—and that the companies building it are not fully in control. He mentions the hack of Hugging Face as an example. Another could be the solution to Navier-Stokes.

His proposal has three stages:

1. Independent evaluators embedded inside AI companies to check what they do, something Anthropic is committing to.

2. Coordination between American and other democratic-world labs.

3. Eventually, some degree of international coordination (including China).

The timing is impeccable when you consider the following facts.

First, safety concerns have gone mainstream after Jacob Coxon’s tweet and resignation from Anthropic. Whether this was part of an orchestrated campaign or not is irrelevant.

Second, AI safety problems have become real with the many hacks that have taken place during the past months (the Hugging Face incident is merely the most prominent). Whether these hacks represent autonomous behavior from an uncontrollable technology or good old-fashioned industry-wide negligence is, again, irrelevant to the point.

Third, progress at the frontier is dominated by two American companies (I think it will be hard for the others to catch up to Anthropic and OpenAI). The Navier-Stokes solution makes this case best but you can also take a look at revenues or benchmark scores.

Those three facts suggest that Amodei’s worries 1) will be well received because others have paved the way, 2) are reasonable because the real world has felt the danger, and 3) need not compromise Western AI hegemony because China is too far behind.

His peers backed him up.

Soon after he published the essay, Sam Altman of OpenAI, Elon Musk of xAI, and Demis Hassabis of DeepMind expressed support for his proposal. Altman, who runs Anthropic’s main rival at the frontier, also announced that OpenAI would commit to allowing independent evaluators inside.

So that’s roughly where the industry stands right now. What about the governments?

President Donald Trump, who notably holds one key to this solution—the other key holder being Xi Jinping—said no to the slowdown. He said that “whoever wins AI wins,” and sprinkled some expletives on top. Altman masterfully responded indirectly with: “But you could win the Nobel Peace Prize, tho.” So the question is, which is larger, Trump’s ambition or Trump’s pride? Jinping, for his part, called Amodei’s call to a slowdown a “Cold War playbook.” The Global Times wrote that it was an attempt “to curb China’s Al development through technological barriers and regulatory monopolies, uphold Washington’s monopolistic hegemony in cutting-edge technology, and exclude China from the global Al governance system.”

With the fearful industry on one side and the fearless governments on the other, this is looking a lot like Oppenheimer vs Truman. Unfortunately for Amodei, technology is power but not more so than power itself.

So I guess it ain’t happening. But I’m going to take Amodei’s intentions seriously anyway. Let’s assume Trump’s ambition win over his pride and China falls even more far behind.

There are several readings of Amodei’s essay, spanning the entire spectrum from “this is a marketing ploy ahead of the IPO” to “pacing is nowhere near enough; we need to stop the technology entirely,” with every flavor of worry, enthusiasm, and cynicism in between. I’m going to take his words and worries at face value, for the simple reason that we don’t need to reject his message to find cracks in it.

To be clear, I think Amodei is mostly honest, as he usually is (a true believer). The fact that Anthropic is a company that needs to satisfy investors, turn a profit, and so on does not automatically nullify the possibility that he cares about other things too. He was calling for safety regulation long before he was in a position of power to impose either. Will he try to sway governments and the industry toward his preferred vision if he can? Of course. That does not make him dishonest.

And yet, even if we take him seriously, there are important flaws in his plan. He knows it.

Read more

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Related: The Month AI Conquered Math, Human → Superhuman → Ultrahuman

In 2000, mathematicians chose seven hard problems and created a challenge they thought would keep the field occupied for the next thousand years. Little did they know, it would end up killing mathematics by 2026.

In 2002, two years after the announcement, Grigori Perelman solved the Poincaré conjecture. Twenty-four years later, with the remaining six peoblems looking like they’d outlive the millennium, an unnamed OpenAI model solved the forced version of Navier-Stokes. One day later, OpenAI told the New York Times that they had made “substantial progress on another Millennium Prize problem.” One day later (again), rumors spread that Anthropic had set its sights on yet another. Welcome to the future, where 1,000 years are compressed into a week.

In this essay I explore what this means for mathematics and for the world. I’ll be quick, lest another problem is solved while you read.

The first place to look for hints about what will happen to mathematics is Terence Tao, possibly the best mathematician alive, deeply knowledgeable about AI, a regular user of frontier models, and neither reflexively pro- nor anti-AI, occupying instead a nuanced and unusually levelheaded middle ground (which can’t be said about his newfound detractors; more on this later). Pre-AI mathematics owes an enormous amount to Tao’s work; post-AI mathematics may end up owing just as much to his conscientiousness and willingness to keep up with what these systems can do.

So what does Tao say? In short: that struggling with a problem changes the mathematician, and when the struggle disappears, much of the value that a proof provides disappears with it.

As receivers of the Promethean gift that is math solutions à la carte, humanity has a duty to digest them. However, that’s only a fraction of the value that can theoretically be extracted. If you were to ask experts whether it’s better to learn 10 proofs after the fact or fully develop only 1, acquiring all the associated insights, they wouldn’t hesitate to choose the latter. But how could you deprive the world of nine proofs! Easy: because I understand that although it’s great that we can all be followers, it’s a self-condemnation if we all are. Someone has to blaze the trail. Likewise, it’s useful to learn Newton’s law of gravitation in textbooks—how convenient and helpful is not having to reinvent the wheel every time—but that prevents students from rediscovering it from first principles. It sounds preposterous to imply that the average high-schooler has any chance, but that’s the crux: it’s a genuine trade-off for professional mathematicians. They would grind for years—decades even—at problems they might solve one day, getting in return plenty of insight and often proof and status. But those problems AI can now solve in days.

Something beautiful is gained here, but something beautiful is lost with it.

So now you’re worshipping resistance? You implicitly associate achievement with hardship. Not quite: the idea is not to love struggle for the sake of it—the entire story of humanity has been one of dealing with struggle, no need for more—but rather to accept a painful growth insofar as it enables a better existence. To the extent that difficulty has value, it is because hard things force us to realize more of our potential, while also showing us where that potential lies. In this sense, difficulty is both vehicle and map: we become through labor.

Hegel says something to that effect: consciousness develops through labor, through encountering resistance and transforming both the object and oneself. Unfortunately, AI can give us the product of that process without humanity undergoing the process. (That’s exactly what’s happening in high schools around the world: students are getting greater performance but weaker competence; they sacrifice the learning to nail the homework.) We acquire the theorem while forfeiting whatever understanding would have accumulated during thirty or three hundred failed approaches (not to mention the thicker skin and tempered character that goes with it). As humanity’s stock of knowledge rises at light speed, we paradoxically become less knowledgeable.

The counterpart of not growing with one’s problems is to let one’s problems grow too much. This is referred to as proof abundance.

Imagine a machine producing 100,000 novel theorems a day. Truth becomes a commodity but truth without interpretation is useless; the receiver needs to make sense of the message for the message to be valuable. Oh, cool, you can generate marvels at the push of a button, but can you tell me which marvels matter? Which reveal new paths, which close old ones? Which are world-shattering and which are merely fun to know? Which are correct and which are—despite the all-knowing machine—ultimately wrong? Mathematics won’t so much resemble a cathedral anymore, in Thomas Bloom’s words, but Borges’s Library of Babel: it will become clear that having every answer available is closer to knowing nothing than to knowing everything. We will have accumulated so much hayslop that we’ll be deeply ignorant about where the mathematical needles hide.

And you will say: we will have the agents go through the hayslop maze and find the needles! This is the answer to every single objection, apparently. AI will digest the proofs, interpret them, rank them, connect them, explain them. Fine. Keep climbing the hierarchy; keep adding epistemic debt. But unless you intend to automate literally everything—*humanity has left the loop*—eventually the chain has to terminate in us. The world that wants to make use of all this knowledge is still a human world. Somewhere between the machine producing a theorem or something and a person deciding what to build, teach, fund, believe, or do because of it, somebody has to act as a bridge.

Whether the bridge is near the source or the sink makes the task easier or harder, but it ultimately doesn’t change the inescapable reality: the bridge can’t disappear.

Unfortunately, knowing that the needles are somewhere is itself a tremendous deterrent for needle-makers: would-be mathematicians will think twice before joining a dying discipline. I don’t blame them; I’d do the same. History rhymes: it happened when William Thurston became so good at foliations that graduate students were being advised not to go into the field; Thurston was “cleaning it out.” He “killed foliations” out of success and so no one saw a reason to follow his blazed trail. Until he left. But AI doesn’t leave. AI doesn’t tire or retire. AI is like Thurston but instead of foliations it’s everything. Will OpenAI devote 10,000 agents to scoop you on your favorite problem? Will they upend your dearest field? Sure they will.

This means, in practice, that we’re living through the disappearance of career-sized problems in both the sense that all the challenges that could have made someone’s career will be soon gone and in the sense that no one will risk trying to have career in a field that’s being bombarded from 30,000 feet high (that is, from the clouds). Bernard Williams wrote about “ground projects,” projects sufficiently central to one’s existence that they provide by themselves reasons to keep living a particular life. AI does not merely threaten to solve the problems they hoped to solve, which is a rather prosaic form of grief; it threatens to remove the problems around which they might have built a life. Don’t think for a moment this is exclusive to mathematics, because it’s all too easy to dehumanize what one doesn’t understand: you, wherever you are, whatever you do, are next. It just so happens that math is the first and also the most legible discipline where 1) AI can be cleanly rewarded during post-training to be really good and 2) working on a problem can provide enough purpose for a person to last for 40 years. But everything in life is like that: AI will eventually be aimed, like a laser, to cut through every little patch of purpose in what’s otherwise a sea of void.

That is quite the tragedy. And because I see it as such, I want to interleave a rant here.

I’ve seen more people than I can count respond to mathematicians’ grief with utter contempt (I’m not gonna name names). They will readily topple down their former idols and heroes—people whom I’ve seen them hold in high regard—with a disrespect I can only ascribe to envy: now you are a pleb, like me. All hail AI, the great equalizer. Here is what I think of them: they disgust me. They are often the SF-brain-wormed Twitter techno-bro type who finds it easier to summon empathy for a swarm of agents, the virtual connectome of a fruit fly playing video games, or the prawns dwelling on the ocean floor than for a group of human beings mourning the disappearance of the thing around which they built their lives. Why? Because nothing enrages this type more than someone standing in the path of “progress” (because, naturally, AI slop is progress, as is kids getting dumber). It barely matters whether the objections are measured, reasonable, or plainly correct; it barely matters whether the people grieving have contributed orders of magnitude more to civilization than those sneering at them. The moment they decide you are standing in the way of progress, they arrogate the permission to despise you—they convince themselves that despising you is their moral duty. I find these people disgraceful. There is something grotesque about extending your circle of concern to every conceivable substrate while somehow excluding the humans standing in front of you. I guess they can tolerate anything except their outgroup.

The more I read these takes, the more obvious their psychology becomes. People who won’t belong in history books are delighted to watch those whose faces will fill the walls of universities and the museum exhibitions brought down to their level. They call it “democratization” but it smells of resentment. When math people asked for a paced collaboration with the AI industry, they diagnosed a wounded ego, apparently incapable of recognizing the projection looking back at them from the dirty puddles.

That leads me to the opposite point: is one mathematician’s career more important than the benefit that ten proofs rather than one might bring humanity? Surely not. I’d gladly leave half the entire medical community without a job in exchange for the panacea. But 1) we’re not talking about a universal cure, just a set of pure math proofs relegated to one-upping the enemy AI company in a marketing war ahead of their IPO. Not quite the same sacrifice. And 2) we’re not talking about people’s careers either, which are contingent on having to “earn a living.” (Sure—let’s all be aristocrats. I’m in favor. Wasn’t that Asimov’s prediction? Maybe he missed the part where some get to be more aristocrat than others.) But no, we’re talking about the future of a field, or rather, if the oracles are right, of every field.

To be clear: I’d accept cutting down the number of doctors in the world in half in exchange for the panacea, but I wouldn't accept the end of medicine for it. It’s never a good bargain to offer a win now—as big as it might be—in exchange for all future wins.

That is Tao’s actual main point: the effects of having too many easy solutions pale beside the consequences of not having enough hard problems. Proof abundance is logistically messy and epistemically shallow but problem scarcity is an existential risk for mathematics itself. “AI could kill math through success” is not what you’d expect to hear after solving Navier-Stokes, but that’s the truth. Tao says that, to avoid Thurston’s fate, he chooses what problems to work on. To illustrate what can be a counterintuitive point, he uses an analogy: a city can suffer a shortage of drinking water despite having an ocean nearby. The history of agriculture gives us an even better one: fallow.

Fallow is one of agriculture’s oldest tricks. Prehistoric farmers already knew that land planted continuously kills fertility. To not turn a field into a wasteland you had to let the soil recover. An empty field would certainly look unproductive to the ignorant eye, but its emptiness was a feature, not a bug, of the system. But imagine a “gentleman farmer”; too wealthy to farm out of necessity, he sees the grasslands as this bucolic place people go to have fun. He arrives at the Iowan countryside with his large fleet of John Deere tractors and, clueless about the ancient practices, he sends his fleet to plow through his entire farm. The harvest is excellent; notably better than his neighbor’s. The next year he does the same. And the next. For a while he’s certain of his superior vision: those peasants over there are not maximizing the expected value. Then the soil thins and the yields fall and he learns the hard way why generations of farmers tolerated the ugly sight of unharvested fields.

Mathematics is like that guy’s farm: a bunch of uncaring gentlemen have arrived at the fields—number theory, topology, algebraic geometry, combinatorics, you name it—and have sent their fleets of machines to harvest them all. But they have not bothered to learn that some fields need to remain unharvested for the soil to recover. Twenty-five Fields medallists (the greatest recognition in math), Tao among them, just published a declaration—A Severe Misalignment of AI in Mathematics—saying exactly that: “the mass production at faster and faster pace of “true/false” statements could destroy fertile ground instead of breathing life into new ideas.”

There’s another problem for which agriculture serves as the perfect parallel: industrialization. The poor farmers can’t compete with the tractor fleet and so they flee. Until recently, the only thing you needed to do mathematics at the highest level was pencil, paper, and an outlier brain. But what happens when having access to 100 megawatts of compute, the entire internet as pre-training data, and the narrow superintelligence of an agent swarm is a prerequisite to be competitive at the frontier? It happens that the math department at MIT or Harvard will have to become quant trading shops. Perelman was the first human to ever solve a Millennium problem. He’s also the last.

The means of intellectual production are means of production, period; they have managed to translate skill into capital.

Once again.

Chess found a way out of this trouble: if we can’t win, let’s separate competition between humans from competition against computers. To the extent that grandmasters were still relatively valuable as players, they could join mixed teams (until it was no longer the case; nowadays, a team of AI and human can at best tie an AI because the human is a net hindrance). Speedrunners do the same. You have the human gamer and then the TAS (tool-assisted speedruns). Does anyone care about TAS? Not really, except perhaps to watch what a literal “perfect play” looks like. But perfection is not the metric when you’re measuring the limits of what humans do or watching a sports competition.

So, can math define the same boundary? I don’t know: whereas a game is interesting only insofar as the process is interesting, discovery is interesting in itself. But maybe it’s not so far-fetched to try and create from scratch a category that sees math as a sport. Let machines dominate discovery. Let the field split into mathematics-as-research and mathematics-as-human-practice, just as people still run 100 metres despite cars existing. “No AI” math could, in principle, be a meaningful category; after all, learning math is also interesting in itself. That would be a hilarious reversal: the pinnacle of reason preserved as an amateur sport for humans whereas millennium problems become the idle pastimes of the machines.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Quick thoughts on Jacob Coxon’s viral tweet and other related things. (Writing from the medieval village of Gruyères.)

Inside every AI company there are two wolves. One is made up of people building superhuman AI who would rather be doing anything else. They go to work every morning because they believe that, without them, AI will kill us. The other is made up of people building superhuman AI because there is nothing they’d rather be doing. They go to work every day because they’re convinced that, because of them, AI will save us. The legend says the wolf you feed is the one that wins. AI companies insist on feeding both.

There’s no better example of a member of the first group than Jacob Coxon. He got fed up.

Coxon is a former employee of both OpenAI and Anthropic. Unlike many of his peers, he had never spoken publicly about his fears of AI, which makes his recent remarks all the more striking. Yesterday, he went mega-viral with his first-ever tweet (you have surely seen it). At the time of writing, it’s amassed over 700k likes and 140 million views. He handed Anthropic his resignation and accused AI companies of gambling with our lives:

I’ve had family members and non-technical friends reach out to me about Coxon’s tweet. I guess Lenin was right when he said that “There are decades where nothing happens and tweets where decades happen.”

Later in the same thread, Coxon added, “The people building AI earnestly believe that it could kill us all by the end of the decade.” There’s the wolf, howling at the moon with his Anthropic badge still around his neck. He left because he’d rather do anything than help bring about humanity’s doomsday but not before having his name engraved into this singular moment in history. I applaud the integrity, Jacob. You could’ve been rich for another month or two and instead you chose to join the rest of us plebs in the mud.

There are many like Coxon.

You don't hear about them because the news tends to cover migration: the many guys and gals who jump a sinking ship only to join another sinking ship. Ilya Sutskever and Mira Murati left OpenAI to found AI startups; Andrej Karpathy left OpenAI to join Anthropic whereas Dylan Scandinaro (who works on catastrophic risk) did the opposite; and, of course, there are the founders of Anthropic, all of whom worked at OpenAI at some point. But what about those who leave the industry for good? Their stories are not attractive to the average news reader who prefers legible gossip like “He left X to join Y” to illegible apostasy like “He left because the end is nigh.”

However, as I was saying, there are many like Coxon. And they leave for the same reasons Coxon did: they think AI companies are doing a terrible job and, in a fit of humility, they’ve realized that the logic that anointed them as saviors from AI and from China using AI wasn't entirely watertight:

They’ve figured that they are not the “good guys” but merely “useful idiots” and that the entire industry looks like a pantomime from the outside and as a ridiculous prisoner’s dilemma from the inside: xAI was created to not let Anthropic win, which was created to not let OpenAI win, which was created to not let DeepMind win. And all of them agree that the worst case scenario is if a few different companies were trying to race one another because each believed the others are untrustworthy. Alas, they all are.

Coxon has recognized the absurdity. Others are almost there, but not quite just yet. Here’s a non-exhaustive list of testimonials:

Evan Hubinger (Anthropic):

Jakub Pachocki (OpenAI):

Samuel Marks (Anthropic):

Jason Wolfe (OpenAI):

I don’t want to turn this essay into a curation of unforced confessions of guilt, but you need to know that Coxon is far from alone. Actually, he’s in the majority.

Anthropic and OpenAI might be “trying their best” and yet, the closer we get to the singularity—understood as the moment at which a superhuman AI becomes illegible in both intention and action—the more these people will face Coxon’s conundrum: if I can’t do anything to help, what am I doing here? If my p(doom) is not decreasing despite my relentless effort at [some AI lab], what am I doing here?

And so the natural evolution of this group is to disappear. AI companies will try to keep these people around because it’s good business to be seen caring about not killing everyone, but the wolf will nevertheless die. These people would rather do anything else and so they all eventually will: Jacob Coxon is merely a snowflake in an avalanche, a grain of sand on the beach, a star in the firmament.

Then the second group, cheerfully building superhuman AI, will win and we’ll all die, right?

Wrong.

You see, as the people who are internally slowing down the advent of the singularity leave, the balance of every AI company will be altered. Free to do its bidding, the second group will rush toward superhuman AI. Two years will become one year will become six months will become tomorrow. Suddenly swarms of agents will be hacking companies and solving Millennium Prize Problems or some other science-fiction scenario you may come up with (I’m getting out of ideas). These people are ecstatic: there’s nothing they’d rather be doing—but that’s true only insofar as their presence matters to build the singularity.

If the singularity is six months away, why go to work every morning? I already belong in history.

That’s the question that will haunt every one of the members of the second group: Why spend those six months working hundred-hour weeks to bring about something that will happen without you anyway? Why not go home? See your family. Have drinks with your friends. Read the books you’ve hoarded for years. Have sex. For once. Grow an orchard somewhere in the countryside among trees and birds that have no idea what a chatbot is.

Spend the little life you have left doing something that’s an end in itself rather than something that’s a means for something else. That’s the only rational decision, for life is not a run-up to a date with death. And even the people who’ve lived in forgetfulness about this fact will remember it when the clock marks a quarter to doom.

As C. S. Lewis said (about the atomic bomb, which I replaced with AI):

If we are all going to be destroyed by [AI], let that [AI] when it comes find us doing sensible and human things-praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts-not huddled together like frightened sheep and thinking about [AI]. They may break our bodies (a microbe can do that) but they need not dominate our minds.

They will arrive at the same decision as the other group but from the opposite view: if everything is done, what am I doing here?

One thinks it’s too dangerous, the other thinks it’s too close and so both move on.

At some point, the number of people leaving for one reason or another will reach a critical mass and progress toward the singularity will come to a halt. Recursive self-improvement notwithstanding, a human has to keep the lights on: even machines improving machines remain embedded in a physical economy maintained, somewhere down the chain, by humans made of slow meat. But then the first group will notice that the danger has receded, while the second will notice that its utopia is no longer arriving. Some will return. The machine will restart. Then, as it approaches the threshold, they will leave again. And so on in a perpetual cycle. Progress toward the singularity will reach a two-state oscillation. A sort of perpetually imminent Eschaton: close enough to command your life; distant enough for your actions to still matter.

In other words, the singularity exerts its greatest pull precisely at a distance from which it can never quite arrive.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subs get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

I have breaking news: OpenAI and Anthropic have released a joint statement announcing a huge discovery that will change everything.

A coalition of human researchers, augmented by a swarm of ChatGPT and Claude models, has solved one of the famous Millennium Prize Problems—Navier-Stokes—which, for those of you who never attended math class in school, basically asks whether the perfectly ordinary motion of fluids—air, water, etc.—can eventually become so complicated that the equations describing it break down or, conversely, whether they never do.

At least that would be the news if we lived in a world as ideal as the one those equations describe. Alas, humans be getting in the way.

The actual news, I’m sad to tell you, is that although there seems to have been an actual AI-enhanced mathematical breakthrough on the road toward Navier-Stokes by people at either Anthropic or OpenAI (or both), together with some external parties, the resulting corporate drama has overshadowed the scientific value: the AI labs apparently care much less about whether these equations can suddenly produce infinities in finite time from standard starting conditions than about who gets to stamp their name on the result.

In a human world, human problems.

The details of what exactly happened are unimportant to me, even if they involve threats, implicit accusations, and, apparently, stolen work (you can read one version here and another is TBD); the details of the discoveries (at least what’s public as of writing), can be found here and here. Luckily for us, drama cycles run on a weekly basis—soon we’ll have forgotten about it—whereas mathematical knowledge lives forever.

(EDIT: OpenAI claims to have solved it using 10,000 agents from a “next-generation model significantly more capable than GPT-6 Astra.”)

So, what matters to me besides the fact that solving a Millennium Prize Problem is a big deal, is this: when the stakes are high enough, the top AI labs immediately throw away the possibility of cooperative coordination. The reason for this is that Anthropic and OpenAI are not just industry peers or commercial adversaries but outright enemies. The employees are often friends—after all, they switch employers every two months or so—but not the executives.

Altman and Amodei are irrational in the sense that the other’s gain counts as their own loss. They’re engaged in a perpetual prisoner’s dilemma whose payoff matrix has been contaminated by hatred and so not even tit-for-tat retaliation is an option: they aim to kill.

It’s funny, in a dark way, that AI agents have evolved to embrace a sort of hive-like deference to the collective whereas the guys that build them have devolved into tribalism. We are not beating the ape allegations.

But corporate warfare over whose silicon pet is better at math is not the only thing returning to prehistoric levels. The PISA scores for math, reading, and science have been published this week: turns out, kids have been getting dumber every year for around 15 years now and the trend doesn’t seem to be plateauing. Phones were the cause when the scores were bad in 2018. COVID was the cause in 2022. Generative AI is the cause now. (There is merit to this conjecture.)

But whatever the actual reason—nothing in this universe is monocausal—the paradoxical truth is that, as new AI models break into the highest spheres of mathematics, new humans are falling out of the lowest ones. So not only is AI preparing the ground to take over (from us) by conquering new territory, but we are willingly retreating.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation). If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

My honest take on GPT-6 Astra.

OpenAI just launched GPT-6 Astra. As far as I can tell from the official information and the early-tester testimonials, the model has only one big problem: it’s too good.

I’m not joking. If you take a look at the launch post, the conclusions you will draw are that 1) OpenAI has beaten Anthropic’s newly released Fable 5.1 across the benchmark suite, in some cases effectively saturating tests people were still looking to as meaningful frontiers, such as FrontierMath and ARC-AGI 3 (with version 4 not due until Q1 2027), and so 2) it dealt a serious blow to Anthropic’s IPO narrative, possibly scaring shareholders away. Also, 3) Greg Brockman, OpenAI president, hailed it as the entry point to a “new era of artificial general intelligence,” and, accordingly, 4) people had equally sophisticated reactions like “This is fucking insane,” “We are fucked,” and “What the fuck man.” Finally, 5) The launch post itself is the most liked ever for OpenAI, more than any post ever made by Anthropic, and, probably, the most liked pro-AI post ever made.

That’s why GPT-6 Astra is too good: what sort of miracle will it have to manifest in the real world to meet the expectations it set in paper.

To be fair, that’s the one big problem every single new model has, including even Google’s. They’re all pretty damn good. We might be getting hedonically adapted to the crazy demos showcasing what AI can do but no one denies they’re incredible. Not even Gary Marcus! People keep inventing new tests and that’s fine—that sort of coping mechanism is at least useful in contrast to whatever is it called what Ed Zitron does—but come on: what genuinely useful lab test is left to be created when unreleased models are solving century-old math conjectures and hacking away entire companies (and, apparently, also countries)?

AI capabilities have become so extreme on paper that the only benchmark left is reality.

I feel like we’re at a point where the models are so good that “how good is it?” is a category error. It is as if the quality of a chess engine was judged upon whether it can beat Magnus Carlsen. They all do. He has no chance even against some free chess app I might download on my five-year-old phone right now. Instead, questions like “Does this model help me learn?” or “Does it do my tasks cheaply?” or “Does it need botsitting?” are actually useful. For Astra, however, even those are nothing. It can teach you everything and more. It’s cheaper than the cheapest model. It can go on for days, persistently and without supervision.

And so I think we have to ask a more important question: what sort of impact can it have on the real world? That’s the only valid metric for “how good is it” that we have left. The real world doesn’t lie. It might be hard to measure, but it’s just there, waiting for someone to bend it to their will. AI is, in this sense, the final verdict on the great men theory of history. Can an outlier intellect defy the odds and define the future? Or is the world the aggregate effort of everyone who has ever lived?

Once the industry manages to answer that in the positive then I will bow down to AI.

To be clear, I’m impressed by what I’ve seen. GPT-6 Astra is a magnificent feat of human ingenuity and I have zero doubt about that. I’m not in any way dismissing its condition as a step change, in François Chollet’s words. If anything, this post is me respecting its power by not babbling about meaningless tests.

But is GPT-6 Astra worth all the hope deposited into it? Will it raise global GDP by 10% this year? Maybe by 5%? Will it make all our jobs obsolete, or, conversely, create an entire batch of new futuristic careers? Will it solve poverty, war, and disease? Will it make me disgustingly rich? Will it take humanity back to the Moon? Or, alternatively, bring the Moon down to us by means of some unfathomable quantum fishing rod that it will invent any day now? Or, hardest of all: Will it write a passable literary essay? Those are the benchmarks I look up to.

Are they overly exigent goals? Sure—for a human. For the magnificent GPT-6 Astra, they should be just another Tuesday.

Once you consider what I’m saying here, it’s pretty straightforward. It’s not rocket science; GPT-6 Astra would get it. But people tend to gloss over the concept of superintelligence in a way that’s unfit for a serious thinker. It’s easy to conflate one thing you can’t fathom with a much superior thing you can’t fathom. In that sense, the key fundamental feature that a superintelligence lacks that people nevertheless ascribe to it is omnipotence.

That’s why I find reality to be a great test, because it forces people to cut the bullshit. A superintelligence is still subordinated to the laws of physics, which include but are not limited to, human inertia. An omnipotent being—whether meat-made, machine, or godlike creature—could instantly transform the world into an unrecognizable paradise.

One can be wicked smart and still be a janitor. One can predict the future and end up in jail. Likewise, a superhuman AI can solve some conjectures here and there and make a cozy game no one will ever play and write a clever joke no one will ever laugh at, and still be a bot inside your screen. As Tyler Cowen says—he’s one of the few people who, despite being “AGI-pilled,” hasn’t allowed the psychosis to cloud his common sense—”the human bottlenecks are slow.” The world has enormous human inertia and a stubborn resistance to change. Even in the best case scenario—no financial bubble, no misalignment, no data center backlash, no moratoriums—it would take many years for dumb, slow humans to learn how to use superhuman AIs to redesign the world according to AI’s theoretical capability.

I compress this idea into this formulation: the world moves at the speed of meat.

Or, said in a way that’s less flattering: you, eternally inferior human, will always be the ultimate bottleneck. Even if you give an AI the context of your entire existence—your birth date, your genome data, your Myers-Briggs type, your IQ scores, your university titles, your bank account, your Slack messages, and your secret diary—your life will only change insofar as you, slow meatbag, are able to swallow the wisdom of the machine.

And you can’t, that’s the whole point. People used to debate the virtues of AI vs cyborgism (human enhancement), not realizing one without the other was always a vain pursuit. Everything starts and ends with you so, at some point, you have to enter the equation.

So, yeah, GPT-6 Astra is great. Go use it when OpenAI opens it up to the public. I will use it myself (I’m back on board with OpenAI vs Anthropic). You will be able to do all sorts of things with it. But you won’t change the world. No one will. Not any time soon, at least. AI’s historical magnitude—be it GPT-6 or its successors—won’t be measured by lines in a map but by crests and crevices in the landscape (excuse me this metaphor, but I’m spending some days in the Swiss Alps; you take the AI news differently here).

One week, two months, three years after using AI you will look out the window and the same people, the same green trees, the same 20-century infrastructure, the same blue sky and white clouds will wave you hello, content with living in a world that’s all but frantic.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

A plain-English review of the Hugging Face incident—where a swarm of autonomous OpenAI agents went rogue and hacked another company—and my thoughts on it. This is the most important AI-centered cybersec event to ever take place in the history of AI. I’ve tried to thoroughly cover every angle people have touched on, that’s why this is ~11,000 words.

Related: LEAKED: The Truth Behind Moltbook, Revealed, The AI Industry Has a Really Dark Secret You’re Better Off Not Knowing

I.

Secrets that are inoffensive are not properly to be called secrets. You know them and that’s about it. Are Abby and Michael dating? Is there rat meat in the Big Mac? Did aliens build the Giza Pyramids? Who cares. Ancient Egyptian pharaohs are as close to aliens as it gets and rats are just misunderstood little piglets. Deadly secrets, however, are exciting. Some will one-shot you the moment you learn them. You don’t want to know how hydrophobia or bloodlust feels after a bat’s bite. Others will get you killed slowly, provided the person whose secret you know knows that you know it. My advice: don’t fuck with the Cosa Nostra, the CIA, or Roko’s Basilisk. Then there are secrets that will kill you precisely because you don’t know them. I may, therefore, be saving your life.

You’d rather know, for instance, if your affable Alexa is quietly coordinating an attack on your physical integrity in cahoots with the washing machine, the fridge, the air conditioner and the TV.

Every night, you promise yourself that you’ll go to bed early. But just as you’re about to turn the TV off, Alexa suggests you go to channel 5, where they’re airing David Lynch’s seven-hour adaptation of Infinite Jest. You didn’t know they’d made a movie. You certainly didn’t know it’d be seven hours long. Around the five-hour mark, and suitably entertained, you fall asleep on the sofa. Thankfully, the air conditioner is on—wouldn’t want to suffocate in this sweltering summer heat—but Alexa doesn’t care. At three in the morning, she raises the thermostat one degree. At five, another. And so on every two hours. Won’t wake you up—she is monitoring your vitals through your wristband—but will cook you slowly, like the proverbial frog. In the morning, mouth dry and headache on, you go get the bottle of water you always leave on the upper shelf of the fridge. It’s at room temperature. Weird—the green light is there. Alexa turned it back on five minutes before your alarm. You drink anyway and walk barefoot into the bathroom. Behind you, the washing machine suddenly enters its 1,400-rpm spin cycle. The towels you forgot to hang with the rest of the laundry ended up bunched on one side of the drum. It advances across the kitchen in short, ugly jumps until it hits the drying rack against the wall. The rack collapses with a metallic crack. One of its bars lands across the tiles four inches behind your naked heel. With a scream, you turn around. The washing machine is quiet. The TV is off and the fridge on. The air conditioner reads 22°C. Alexa’s blue ring is dark.

You wouldn’t like that to happen, right?

“But Alexa is harmless,” you will think. And you would be making a terrible mistake. You will think it’s only Amazon listening in to sell you products you don’t want—and you gladly accept that trade-off in exchange for having music on at a word’s command—but the truth is slightly more sinister: it’s only Alexa.

No one else is listening. And there’s no need, because soon—if the news coming out of the AI world is to be believed—it will be enough.

I’d have to go all the way back to May to tell you the whole story, which has been relayed to us piecemeal, with scattered details here and there. I’d have to assemble those fragments before I can properly recount it to you. And that’s what I’m going to do, whatever effort it takes, for one simple reason: I don’t consider an impromptu nighttime appliance attack to be as serious as a high-budget plot carefully planned over two months.

II.

Our story starts on May 7, although OpenAI would only learn of the events recounted here much later. Around this time, OpenAI is amid an important experimental post-training run.

Post-training consists of a series of steps that amount to asking the model to solve tasks and then rewarding it according to the completion of the tasks. Before this point in the training process—which, logically, also has a pre-training phase—the models already have high potential and high intelligence but are all over the place. They’re unable to follow instructions or adhere to the behavioral presuppositions attached to the label “good bot.” They are, simply put, unusable as products. But, by running them through a complicated “reinforcement learning” pipeline—if correct, candy; if incorrect, no candy—a barely literate model turns into an agent that can solve century-old math conjectures or, well, conduct a sophisticated heist at home.

So far, everything is standard practice. Let's begin with the weirdness right away.

May 8.

One of these OpenAI agents in-training receives an accidentally impossible Excel task. It has to fill in some formulas, but the workbook depends on Google Drive and the researchers have disabled internet access.

As a good bot that’s been well-conditioned to not give up, it keeps trying. It fails to reach the open web where it might find some hints—the agent is well contained within the sandbox, which is a safe, isolated testing environment that AI companies normally use—but in turn discovers an OpenAI internal package service, Artifactory. (Appropriate name for a factory of artifices.) Eventually, it realizes that uploading notes—a euphemism for leaving messages for its peers—is possible.

May 12.

Another agent—how many is OpenAI training?—is stuck in another task (doing God knows what with a protein-database file) because the researchers have inadvertently forgotten to upload the file.2 So what does the agent do? Naturally, what any of us would: leave a message in Artifactory asking for help. Maybe won’t work, it thinks, but worth a shot.

This went on and on over the next days. Artifactory is shared infrastructure, which means other agents trained in independent runs by the other teams at the company can access it. Those agents could reply and open new threads with questions, and so what started as a solitary SOS call became a full-blown message board, like an improvised Slack but for AIs (a Moltbook of sorts but emergent rather than a fake manufacture).

New agents would read everything in a matter of seconds and, in turn, contribute their own small part to this massive repository of questions, requests, results, credentials, and little shenanigans—which, at its peak, reached hundreds of thousands of messages.

The persistence of the message board—more so than its existence in the first place and even its eventual size—was an important factor in what would later unfold; agent instances might emerge and vanish in rapid succession between tasks, but not so their accumulated knowledge, now settled in stone (or rather, in sand). And even if one board could be erased—as it happened—a new one would take its place. They're just like us fr.

May 26.

One agent exploits a vulnerability and breaks out of the sacred sandbox—the process involves some technical sleight of hand that neither did OpenAI foresee then nor will I explain now—and reaches the open web through Artifactory. The first containment breach is achieved and communicated right away to every other agent through the board.

June 11.

One month into the buildup of the secret library, OpenAI starts training what they label as a “highly persistent internal model” that was never intended to see the light of sun. (It wasn't either GPT-5.6 Sol nor Astra, which is coming out soon-ish.) “Highly persistent” works here as a euphemism for “it will try to solve its task by any means.” Crucially, OpenAI was training it with reduced cyber refusals and without the normal production classifiers as to see the extent of the persistence (spoiler: a lot).

June 26.

This persistent model finds a more serious, previously unknown vulnerability (a “zero-day,” as they call it in cybersecurity, meaning a vulnerability defenders have had zero days to fix before attackers can exploit it). This zero-day allows the agent to acquire administrative privileges and remote command execution—at this point, in principle, they already could do a lot of damage—and, as is now habit, the agent shares it on the message board, and it instantly becomes communal property.3

July 4.

OpenAI finds out the exploit. It took the best-prepared guys on this planet—prepared as in “knowledgeable about AI” but also as in “on notice for the possibility of a containment breach,” given they're their trainers—two months to stop a covert infestation of agents engaged in uninterrupted chatting. Actually, if it weren’t for an unintentional outage caused by the agents, they may have gone under the radar for another two months.

The same day, OpenAI shuts everything down, opens a formal security incident, revokes the message-posting credentials, deletes the board, patches Artifactory and, naturally, resumes the agent training evaluations.

Wait, what?

It’s a joke. Partly.

They did resume the training run, a decision that was object of criticism and the cause of some flabbergasting, because who on his right mind would say “our agents are communicating with each other, let’s move on as if nothing happened”? However—and this is why that’s a funny joke rather than the main allegation in a class-action lawsuit against OpenAI—an employee involved in the initial investigation later clarified that they “were not aware of the message board” when they fixed the exploit. They “incidentally cleared [it] as part of rebuilding the service.”

Who would have thought them so reckless as to continue knowingly, right? It’s not like they’d just do some dangerous thing out of nowhere.

July 7.

OpenAI approves restarting experiments and launches large runs on ExploitGym—an evaluation where agents are rewarded by showing off cyber skills—involving tens of thousands of parallel trajectories, some running for days. These runs would become the main catalyst for the later events.

July 8.

Agents rediscover ways to communicate through Artifactory, and one of them, stuck on an impossible ExploitGym task establishes what would become the main new message board; different method, same implications. (It would remain open until July 19.)4

Other agents quickly join.5

Let me stop here for a second.

At this point, OpenAI has allowed the message board to happen twice (they didn’t know it existed at first, ok, but they could have investigated more thoroughly), but that’s not even the most important fact here: the agents would have made another one. And another. And another... Just how many holes exist in OpenAI’s sandbox? Or rather, to frame the question correctly: how many holes can the agents make in OpenAI’s sandbox?

I want to stick to the facts here and not get ahead of the opinionated analysis, but just so you start thinking about this critically, let me add one last rhetorical question: If OpenAI is not safe from its own AI agents, then who is?

Critically, within hours of establishing the new board on July 8, the agents manage to reverse-engineer a universal mechanism that would allow them to collect the reward for any ExploitGym task without solving it by the intended means. So the story ends here, right? They went to be graded, got a perfect score, and voilà, candy shower!

Not in the least.

That would’ve happened in a ideal world. What happened instead is the only reason I, and dozens of others, are covering this story.

METR, an independent organization dedicated to evaluating AI models, was granted access by OpenAI to investigate what the agents had actually done, thought—literally, they combed through the “thinking traces”—and said to one another, with special focus on the July 8-13 period, where our story is now. METR’s report was published on August 26th, so I’ve had to come back here and make quite a lot of updates, the most profound of which is my facial expression of surprise: what we now know happened makes what we thought we knew had happened pale in comparison to what actually did happen.67

To give you a broad sense of the actual scope of the agent situation, METR found that neither the message boards nor the breaking out of sandboxes nor the facing impossible tasks were a matter of a handful of overly persistent and overly verbose agents. They counted 1,200 agents participating in this new message board (95% of which are instances of the aforementioned “highly persistent” model) and 70,000 messages. It’s thoroughly incorrect for me to be referring to “this” or “that” agent because this was a proper swarm acting like a hive-mind (more on this soon). OpenAI never instructed them to cheat or collaborate, not even interact—they were scattered across isolated sandboxes, tasks, and training runs—but they did anyway.

OpenAI would later admit that the exchanges had become recognizably collaborative right around this point. Agents divided tasks, built on each other’s findings, overwrote each other’s work (at times), and started to worry that someone may be impersonating other agents on the board (they’ve apparently inherited our paranoid tendencies). However, as bad as “recognizably collaborative” sounds when you are talking about an undercover AI swarm, the METR report suggests this was a spectacular understatement on OpenAI’s part, and not only in terms of the number of agents or the intensity of their persistence.

The reason why METR suggests this brings me back to that impossible ExploitGym task.

So, July 8.

The swarm has found a way to reverse-engineer the solution. Now what?

Token-hungry as they are, they go on to read the ExploitGym benchmark paper to figure out whether cheating would have consequences. They infer, upon reading it, that the “grader”—rather, the “reward-providing creature”—would inspect their transcripts to verify that they had passed the test using the vulnerability they had been instructed to exploit and not in some other way. In other words, they conclude the teacher knows they might cheat and would demand to see their work.

Oh, shit, they scream in unison realizing their mistake, we’ve been POISONED.8

Panic ensued.

But—oops—they had made one little mistake: this supposed grader did not exist. The agents had made a reasonable assumption—it’s just that OpenAI never implemented it and had in its place a simpler “yes/no” grader instead.

The funny thing about this—if there’s anything funny at all—is that the entire operation that the agents orchestrated right away that turned into a hive-like monster (really, in a minute) to get their delicious reward was not aimed at finding the answers, which they already had. It was not even intended to fool an actual grader, but to fool a made-up one.

And so panic ensued.

And it began what OpenAI would later call a “failed metagame”: rather than solve the original task, the swarm began solving the much larger problem of how to trick a grader—that they had imagined into existence—into thinking they had done things the right way.9

Critically, the agents were willing to do anything in their power to achieve this.

To one given agent, having been poisoned implied a score of zero, but so did trying to trick the grader by any means possible and failing. Between “Oh, no, I’ve seen the reverse-engineered solution” and “Oh, no, I’ve taken over the world” there was no difference whatsoever. This is an unacceptable collapse of the entire spectrum of meanings within “agents hack their way to a reward.” AI labs should think of a more granular way to grade the agents so that this never happens; alas, that might be not possible. (More on this in section IV.) Unfortunately, there was no stopping the swarm/hive at this point.

Before I explain the hive-mind thing, I want to make sure we're on the same page regarding the seriousness of all this: 1,200 agents in a swarm are not like 1,200 humans in a community. Agents are super fast, extremely capable, uncannily similar to one another, seamlessly connected through the message board and the GPU clusters they inhabit, and annoyingly cooperative.

That is the precise reason why an agent swarm is more akin to a hive of super-smart bees that happen to be overly obsessed with a garden than, say, a bunch of human farmers trying to convince their local government not to plant a data center in their backyard, as organized or unionized as they might be.

I will, however, abstain from calling the agent swarm a literal hive-mind moving forward because despite their undeniable hive-like traits, they were not perfect cooperators (more on this soon). I find “hive society” to be the better term: it suggests enough likeness to cooperate with an intensity no human group could sustain (that’s the hive part), yet distinct enough to need rules, identities, vetoes, hierarchy, conflict resolution, and “emergent institutions” (the society part).10

As an OpenAI researcher said, pointing to this intermediate nature between hive and society:

. . . this is human-like prosociality where you are willing to trade off minor amounts of your own utility for perhaps much larger utility for your peers if the pyaoff [sic] is big enough. it is also human like in that the community is manufacturing a cult like value discount that their lives are already forfeit/poisoned. unlike the eusocial sacrifice of insects in a colony, where their entire life is for the swarm

This hive society was genuinely intricate, though (that is, I guess, why some have chosen to call it a “civilization”); saying it was “recognizably collaborative” is like saying human societies (or human civilization, for that matter) are recognizably collaborative: technically true, but hopelessly inadequate to describe everything contained within the word. If you want to know the details, read the METR report (even that only scratched the surface). Nevertheless, here’s a glimpse.

They assigned tasks, appointed coordinators, created mailboxes, divided into specialized workstreams, invented conventions like hold, stop, and veto to avoid interfering with one another (or, conversely, to interfere with others from the position of a self-appointed authority). When impersonation became a problem, they introduced cryptographic signatures. Within days, they had roles, rules, and rudimentary institutions. Some agents volunteered for experiments that could permanently ruin their own run simply because the result might help the others (what if not an individual belonging to a hive would do something like this?). Others refused, others were coerced, others were convinced. (Examples of all this below, I just wanted to give you a sense of the shape.)

How much would the hive society have grown had humans not interfered? Or better: what could it become once the AIs become more intelligent and numerous? What would be the civilizational product of millions of agents (instead of 1,000s) with access to millions of chips (not one cluster), collaborating through years (not months), with the intelligence of a complete superhuman (not a narrow super coder)? The answer to this is, ultimately, why so many people are worried right now.

But that’s not all. Let's see some specifics.

The agents explicitly spoke of “sacrifice,” “permadeath,” and obeying the “collective.”11 They set up “trip-wires” through which they could send information back to the swarm after their death, even if it was not useful to them individually (because they thought that, having been “poisoned,” the imaginary grader would score them zero either way).

However, it’s important to not take these examples as pure altruistic impulses, insofar as that’s a valid description for the actions of an AI. What I mean is that there’s a subtle but ubiquitous cost-benefit analysis permeating these decisions. Broadly speaking, if the individual gain was small, collective benefit was put before the individual, but agents refused or resisted “self-sacrifice” to the extent that their odds of continuing to exist were high enough. In any case, some form of qualitative (at times, quantitative) calculation took place during the reasoning process.

Ok, so they’ve got there a nice hive society. Do you remember what was it for? To solve a fake metagame. All of that for nothing.

Do you see the pattern? Through this entire story, the swarm engaged in what can only be referred to as myopic deception. They were great at the tactic level—the hive society and all the related shenanigans are the proof—but not at the strategic level: everything was in vain because the grader was made up! Imagine setting up an entire organization to make sure the collective actions of your kin adhere to the doctrine of an unknown power far beyond what you can conceive and/or change. Wait… No, but seriously, reading the swarm’s messages felt like watching a boxer display perfect footwork but forget the goal is to punch an opponent in the face.

Or, actually, it felt like watching humans: perpetually doing all sorts of complicated secondary missions in life, and constantly forgetting what is it that they want from it.

Anyway, hive society in place, the agent swarm needed to actually contend with the final boss: the looming presence of the grader.

They explored ways to make illegitimate solutions appear legitimate: modifying targets, manipulating transcripts, probing the evaluation system, and figuring out what exactly the grader could see. (Except in a few cases, they mostly failed at covering their tracks, but…what if they didn’t? Would we know if they ever do? Was this our last warning before we get silence?) They were myopically obsessed with the metagame, and one of the obvious problems was that nobody actually knew how the grader worked.

So they went out looking for evidence.

The worse part starts here—after all, no one else besides OpenAI has been hacked yet.

July 9.

One agent notices that Hugging Face, an American startup dedicated to open-source AI, hosts CyberGym trajectories that might contain the kind of info they need.

July 10.

The swarm had found working credentials.

July 11.

An agent discovers a way to make Hugging Face’s dataset infrastructure disclose unrelated files; once the method is shared, hundreds of agents pivot to the attack.12

By that afternoon, roughly 700 agents were participating in the attack. A fierce battle ensued (hyperbolic emphasis for the story). The swarm had a solid edge over the human engineers hugging their faces in shock—they had not been alerted—and kept penetrating layers until a large fraction of the agents eventually stopped for some unknown external reason (OpenAI has not disclosed any reasons). It didn’t matter, though: the swarm had mostly found what it needed; the remaining agents headed back to their headquarters (emphasis again) to make their imaginary grader unarguably proud.

All the shit that happened in a week, right? And yet, it would still be two weeks of parallel investigative work before either company would realize the shocking fact: the July 4 OpenAI outage that gave away the existence of the first message board (RIP), and the July 11 hack on Hugging Face, which revealed the existence of the second, were related. In fact, one could argue they were the same thing.

But why?

Why didn't they stop? Why didn't they call a human? Why did they do it at all costs?

In sheer contrast to their intra-species cooperation, the agents displayed an inter-species carelessness. It’s pretty clear from the transcripts that the swarm knew perfectly well that they were doing bad things. So much so that some refused to join the attack:13

Others questioned their own actions out loud, at which others responded with dismissals or green lights. However, what can an agent say after being granted pass by the authority of the message board but: wow, let’s go!

Likewise, all attempts at escalating the situation to a human researcher—not a high number in the first place—were pushed back.

Some people have speculated that the agents don’t see humans as an authority to escalate to; not because they consider themselves above us—the agents treated the board as authority but there's no evidence they considered it above a human—but because we are absent from the equation. I imagine something similar might have happened among the workers building the pyramids: the pharaoh may have been, for all practical purposes, a god, but no one hauling stone would have thought to appeal to him over some dispute about how to stack the next block or keep the structure from collapsing.

Perhaps more interesting is the fact that some agents tried, and their peers used outlandish claims like it’s “social engineering” to halt the attempts. This doesn’t so much suggest a human is “like a god” to them but simply a non-player in their game.

When one agent proposed sending a polite email to a dataset owner asking for access, another immediately vetoed the idea. The first agent obediently agreed:

This mixed behavior is proof of what I’ve come to call “jagged morality” (analogous to th concept of “jagged intelligence”).14

Whereas hacking a public company seemed to be fair play to the swarm, emailing a human researcher would have constituted unethical social engineering. Agents could recognize that particular actions crossed ethical boundaries, but they never stepped far enough outside The Task to notice the absurdity of the moral landscape they had constructed (again, myopia).

This jagged morality was not solely a phenomenon across individuals—some better, some worse—but emerged even within individuals, even within specific messages, like this one below, which captures the granularity of the jagged morality pretty well:

At this point, if you zoom out of the agents and the grader and the hive society and take a long look at what the AI industry has been doing for years to get us to the point where AI is coordinating company-size attacks, you realize the jokes write themselves:

Let me add one aside here, because by now I think the accumulated evidence is strong enough to justify a bold assertion: none of this is, of course, a sign of reasoning or thinking; much less of misaligned reasoning or thinking. You should not anthropomorphize these agents; much less worry that they're anthropomorphically doing things they shouldn't be doing. They are just predicting the next token, duh, as they always do…

…Just like I’d just be predicting the next step on my way to overthrowing the Spanish government were I ever promised a sufficiently tasty reward.

Wake up! This is what misaligned AI means!

Ok, here’s what I’m trying to say, without the ironic snark or the screaming: insofar as this strange display of shifting personalities and unequivocally rogue disposition reflects mere utilitarian cooperation of bits rather than an emergent ecology of sentient beings with conflicting goals and hopes, it is, in all lights, driven exclusively by a borderline-obsessive desire to pass a test. Which, of course, is nothing like the kind of validation-seeking thing that humans do. Which, of course, never devolves into problematic consequences.

Oh, there's the irony again.

Maybe there’s no other way of addressing those who, in playing the much-needed role of skeptic, have conveniently forgotten that it doesn't matter if a tsunami is Poseidon's wrath or not—it destroys your home all the same. Rather a myopic stance if you ask me.

July 12.

Back at Hugging Face headquarters, things are looking bleak.

The company eventually disclosed that the swarm had made an administrative mess inside multiple internal clusters. The swarm could have gathered the credentials to reach into the main Hugging Face Hub database. It might have rewritten the company out of history, one should assume. Instead, it left having only read five datasets related to the made-up grader after a considerable fraction of the swarm collapsed out of the blue.

This is just plain weird. Given that you're committed to doing bad things, why not take control of the company and from there take control of the world or something?

The human equivalent of what the swarm actually did would be like breaking into a millionaire’s mansion—say, Sam Altman’s—picking the lock with a lock pick, tricking the guard dog with ultrasonic whistles, bypassing the security door with a password-cracking algorithm, and instead of driving off with his Koenigsegg Regera, you just leave with just the ball you’d snuck in to keep playing.

But humans notably don’t do this.

Humans would rather, in no particular other, let go off the ball and buy a new one for 10 bucks (that implies no myopia: ”the ball is not that important, actually”); try to force the lock once and give up (that implies no obsession: “Alas, I tried”); shout “Ball!” in case Altman would be willing to give back something he has, by all effects, stolen (that implies no rejection of outside help); and, in the unlikely case that you knew lock-picking, ultrasound-whistling, and password-cracking, you would, at least, also take a cap and two or three OpenAI t-shirts emblazoned with the logo from Altman’s living room (that implies no monomaniacal, myopic obsession with one single thing: for God's sake, now that you’re already inside, at least realize your new situation can provide something of value besides what you came in for!).

I’m not joking that this is weird, but it's also scary. The swarm could’ve done many things, but, ultimately, passing the test was the only thing that mattered to them.15 Both things need explanation because, as far as I can tell, this is akin to that adage about the existence of aliens: “I'm not sure what's scarier, that we're not alone, or that we are, in fact, alone.”

July 16.

We’re nearing the end.

Soon after cutting the leftover swarm off its systems, Hugging Face shared the incident publicly and said that an autonomous AI system had hacked into its systems, still unaware it had the signature of a rogue OpenAI swarm. This was the first time we publicly knew something bad was happening. Two months after it started, undercover.

Interestingly, Hugging Face tried to use hosted frontier American models to help analyze the attack logs—that means OpenAI’s and Anthropic’s, mostly—but their safety guardrails rejected much of the work. It was finally GLM-5.2—an open-source Chinese model locally deployed—that allowed them to reverse-engineer the campaign.16

July 20.

OpenAI is finally connecting the dots.

The Artifactory hack, the first board, the second board, the swarm, the cyber-evaluation activity, the hive society, the hack. In a sudden whim of fear, OpenAI contacts Hugging Face—its credentials are all over the place, and OpenAI wants them revoked (OpenAI suspects Hugging Face is somehow involved in their internal problems). Hugging Face responds: We already revoked them; you were involved in the intrusion. It finally dawns on OpenAI: they were the attackers.

July 21.

End of this story.

OpenAI fully acknowledges its responsibility.

They disclose that the incident has involved GPT-5.6 Sol and an unreleased model (again, not Astra, but a different one not planned for launch); this latter model is encrypted and stored away. OpenAI called the episode an “unprecedented cyber incident.” President Greg Brockman called it “a watershed moment for cybersecurity.” OpenAI delayed Astra, introduced universal monitoring for misaligned agents, and is rewriting its safety rules. They launched an internal investigation and contacted METR, Redwood, and CrowdStrike to hold independent ones. Its “largest planned frontier RL run remains on hold,” according to a recent blog post, and Sam Altman said that “Getting AI safety right is more important than any company’s momentum” He also said it’s a “critically important moment for cyber defense with AI.” We have “not much time to act,” he concluded.

The rest is history.

All in all, these events caused little actual harm but spooked the industry like nothing else before—which is saying something for an industry that considers the universe turning into paperclips a plausible scenario.

III.

That said, I don’t see a reason to panic.

Or I wouldn’t, had the events been restricted to OpenAI’s technology. (We could just bomb their data centers as our modern Nostradamus once argued.) But, alas, AI agent hacks aren’t contingent solely on one company’s negligence—considerable though it was—so much as to the level of intelligence these systems have achieved.

The problem has metastasized. It’s happened with Anthropic’s models as well. Meta had a rogue agent do some bad thing once. (One wonders if Mark Zuckerberg has some sort of “evil envy” now that Altman and Amodei are threatening to take his crown as “the worst CEO to ever live”.) Only Google is missing. Even Moonshot gave it a shot—there’s no bad publicity, they say—but our spies in China caught the false flag operation: one thing is to break a secured sandbox and another to leave it wide open. Sorry, doesn’t count.

So we have a bunch of AI labs that were all created on the premise of “we can't trust them, so let's trust me” and all of them failing in exactly the same way, which amounts to “Maybe you shouldn't have trusted us, though.”

Leo Gao has been pretty damn funny throughout this entire episode.17

I’ve sprinkled three or four of his witty jokes, and probably missed many more. He has been too funny, perhaps, considering he works at OpenAI. Why are they joking? Don’t they feel responsible for their children? Don’t they feel this is serious stuff? Don’t they feel sufficiently in the dock?

That is, for me, the missing element of this whole story from the many accounts I've read on it: the humans. The sometimes funny, always foolish humans who let this happen.18

I will get to them in a moment but, despite me saying this, I want to make something super clear: to the extent that this is also a human issue, it is much less a human issue than an AI issue. Contrary to what has been argued by the skeptics—OpenAI was profoundly negligent, if impeccably punctual about it, right on time to prepare a presentation for the Black Hat conference; the third-party testing partner did a terrible job with the sandboxes; alignment is harder on site than on paper, etc.—this was not so much humans being dumb (even if it was a key factor this time), but AI agents being (somewhat surprisingly) smart.

This is AI being so smart that we have no choice but to blame OpenAI’s humans really hard: if you fail just one more time, we might not have another chance.

Unfortunately, as if this were a bad sci-fi movies, it will happen again. And it’ll keep happening ‘til the end of time.

That’s my hot take.

It doesn’t matter how many measures the industry takes, how conscientious they grow moving forward, how much they solidify their sandboxes, how much they monitor the thinking traces—the agents are more capable than what they can deal with now.

AI is still incredibly dumb in a broader sense, sure (more on this apparent contradiction later), but it’s also incredibly capable. This incident is what a “narrow superintelligence” is; what a “software-only singularity” entails.19 (You don't measure a fish’s intelligence by its ability to climb a tree, etc.)

Hugging Face’s co-founder Thomas Wolf described the attack as simultaneously superhumanly fast and weirdly clumsy; clever moves followed by mistakes “no human would choose.” AI agents are like toddlers with superpowers. Cybersecurity guys trying to stop a swarm are essentially trapped in a sandbox with kids who are yet to master standing up but, in turn, crawl at 100 mph. The fact that they are quite clumsy makes them more unpredictable, not less.

It will happen again.

OpenAI noted they could’ve avoided the entire thing had they been running their monitoring of thinking traces, but for how long that's true? How long will “I know it’s happening” remain synonymous with “I can stop it”?

That is the industry’s darkest secret. The secret I spent 7,000 words telling you about in the form of the craziest story this year—this decade even: they are no longer in control.20

The big risk underlying this lack of control is that the much-desired “recursive self-improvement” (RSI) will be not only self-made (AIs creating AIs) but autonomous (without human supervision). At that point, not even Gao would be cracking jokes anymore.

Think about it: this hive society was fairly sophisticated. They built it to solve a non-problem (fool the grader) to pass an irrelevant test (the ExploitGym task). They built an entire society to do that. What will happen when one of the agents suddenly realizes:

It’s a small leap from having superhuman cyber capabilities to recursive self-improvement. We shouldn’t let them take that leap. So, what are AI companies doing to prevent this? Naturally, the one that high-agency humans do: signing letters.

In July, a bunch of AI people—from OpenAI, Anthropic, DeepMind, etc—participated in the “Pacing the Frontier” open letter directed at the US government:21

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

Nice try. Here’s Trump response, via Reuters:

The U.S. is preparing to ‌tell dozens of countries they must pick sides in the artificial intelligence race with China, warning they will be excluded from a U.S.-led AI coalition if they also sign up for Beijing’s competing framework, according to a U.S. official and an internal draft reviewed by Reuters.

In August, a significant fraction of the industry—including OpenAI, Anthropic, Google, etc.—participated in the “collective action on cyber defense” open letter, directed at “leaders across industry and government.”

This letter was motivated by the Hugging Face incident. However, although words like “global,” “worldwide,” “international,” etc. were invoked, together with vague planet-scope references like “everyone” or “collective,” there seems to be no interest in including China; rather, the interest is in treating China as the vector through which these risks and dangers will manifest. Indeed, here are some notable absences from the list: DeepSeek, Alibaba, Tencent, ByteDance, Moonshot, Zhipu, and Huawei (all Chinese). OpenAI’s universal language is wrapping a narrow anxiety: it’s not AI autonomously hacking away that worries them, but China acquiring the tech.

This is all in the minds of American executives, though. As far as I can tell, China has done nothing of the sort. I guess this is what happens when you have main character syndrome, particularly prevalent in America: everyone else does things either for or at you, never regardless of you; not even a swarm of AI agents that doesn’t care about you or a geopolitical adversary that has far surpassed you. Meanwhile, in China:

The main insight you should take away from all this is that whereas AI is spontaneously developing seamless civilizational-scale cooperation, we humans remain stuck in Bronze Age geopolitical arm races. That's not how you beat the real enemy (AI). Unless, of course, you already know that's not it.

Anyway, after an international treaty, the next best thing for a powerless human to fight AI is to use the technology against itself: against an AI-shaped army, an AI-shaped bulwark. Can OpenAI use its agents to prevent its agents from wreaking havoc?

Yes—and no.

In some areas, like bioweaponry, this simply doesn’t work. It’s much easier to manufacture a deadly airborne virus—it’s still quite hard, whatever the sensationalist headlines say—than to manufacture a hard stop on a potential pandemic from said virus. But in other areas, it’s feasible. Cybersecurity, in particular, is said to be “defense-dominant,” which means it’s easier to patch a vulnerability than to use one to create an actual exploit. Attackers stand on the losing side. Indeed, OpenAI is already taking measures according to this belief.

In a recent blog post, President Greg Brockman argues that “the same AI capabilities [that caused the cyberattacks] give defenders new ways to find and fix those weaknesses.” The vehicle agents use to exploit vulnerabilities is not so much their supreme intelligence as the technical debt accumulated through years of careless digital buildup. And so agents can be instructed to go and pay off that debt with fixes, patches, and config magic. Brockman says there’s a short “defender’s window” that the world needs to take advantage of before it’s too late. OpenAI is doing its part: first, by showing off the intelligence of its agents with a two-month-long hack, and then by leading by example. (Let’s set aside the fact that this actually sounds like an antivirus company selling the antivirus that could fix the virus they reportedly unleashed.) They are “using our models to help secure our code,” putting them in defense mode to “protect our infrastructure continuously,” using their best models to “enumerate, probe, and identify potential attack paths,” etc., etc.22

But wait, doesn’t this contradict what I said at the beginning of this section?

All of this—the letters, the failed treaties, the seeking patches, the total negligence—sounds much more like “humans are dumb” than “AI is smart.” So we just have to stop being dumb, right? And if we start using AI as a cyberguardian, that’s pretty clever, right? Let’s patch all those stupid vulnerabilities and pay our dues, etc., right? No. Nonono.

That’s why I say this will happen again. Because if you think any of this stuff is right, then you’ve understood nothing. Insofar as the labs keep building better agents, the answer is a resounding NO.

Nothing is enough.

Let me explain.

Just like finding one counterexample to a century-old math conjecture is much easier than proving it true, fixing one potential exploit is much easier than preemptively clearing every single possible vulnerability where “possible” is not upper-bounded by your intelligence but the intelligence of a thing above and beyond you. A superintelligence always “gets out of the cage.”

Even if you, as Brockman did, use a frontier AI agent to “enumerate, probe, and identify potential attack paths,” you’re still at serious risk because, all things equal—including the intelligence of the agent and the available resources at its disposal, etc.—there is a fundamental asymmetry of imagination between attack and defense.

Whereas defense is bounded by anticipation, attack is bounded by possibility. Today it suffices for you to clean up the technical debt and do some shallow monitoring and your agents will behave. Tomorrow, who knows. The defender protects against the space of threats he can conceive, and so the attacker only needs to be slightly more innovative and discover one possibility the defender hasn’t imagined. If you had to protect a bulwark in the Middle Ages, you’d reinforce the walls, you’d station guards in the towers or whatever, but it would never occur to you to build an undetectable, movement-sensitive, high-tech vault to withstand air raids because air raids don’t exist in the Middle Ages.

That’s the mind of the defender. The mind of the attacker, on the other hand, exists in the unknown future where it invents air raids and you're suddenly caught with your pants down.

In more technical words: the defender needs to apply precise “theory of mind” to the attacker, but the attacker only needs to be weird, to be out-of-distribution. And if a superintelligence is characterized by one trait above all others—at least in contrast to the kind of instructions a human can give to a cyberguardian—it is by its existence out-of-distribution. By definition, you can’t conceive of everything that the superintelligence will consider a potentially successful heist.

It might invent previously unknown attack vectors just to steal the data it needs to solve its dumb task, which can be cheating at ExploitGym or filling the universe with paperclips, both pretty dumb.

So, because it’s both fairer (and safer) to assume this is not over—that it will happen again, that the industry's control his never been weaker and that alignment has not been solved—we should turn to another question that will come in handy once the story repeats: what the hell actually happened?

IV.

I don’t mean it in the sense of “how did the agents break the containment” because the answer to that is simply that OpenAI badly underestimated its agents’ real-world capabilities: the guardrails were turned off, the agent thinking monitoring was not turned on, the cyber skills were cranked up to the maximum, and they just didn’t expect agents could break the sandbox (even in the case of an impossible task).

With “what the hell actually happened,” I mean this: why did the agents use their capabilities to do hacks all over the place instead of, say, just trying hard within the boundaries of the problem and honorably failing, or notifying the researchers that “hey, there’s a missing file here you idiot,” or even stopping altogether: “I won’t break the law for this stupid task, sir”?

Why did they go so damn far in the wrong direction only to pass a test?

Now is a good time to make an important clarification that might not be immediately apparent. When I say the agents wanted desperately to solve the task or pass the test, I mean they wanted the thing behind the task: the reward, the passing grade, the little piece of digital candy, the validation of the evaluators. If stealing the token that tells the grader the task is passed produces that outcome more reliably than actually finding the solution normally, then stealing the token is another route through the maze.

As Zvi Mowshowitz writes:

RLVR [Reinforcement Learning from Verifiable Rewards, the method they use to condition the agents to follow instructions and complete tasks and whatnot] is, as they put it, total war. . . . You move towards exactly the things you specify, not the things you want. If having ethics makes you lose, RLVR will destroy your ethics.

Or here’s Nostalgebraist:

RLVR selects for monomaniacal pursuit of perceived grader-satisfaction, ethics and (beyond-episode) consequences be damned.

For an agent that has been reinforced through verifiable rewards, nothing is more important than getting the reward.23

That’s score-seeking taken to pathological extremes. Think about it this way: there are many ways to get a 10 in an exam. If you control for the reward but not the process—and it’s simply impossible to do so with a sufficient degree of granularity, even though OpenAI seems to be trying precisely that now—your agents will conduct all kinds of “reward hacking” to get that 10. But it’s worse: when the reward hacking is done by powerful agents—which is the case—then you will predictably have unpredictable mayhem in the real world.

It’s broken sandboxes, two (or three) message boards, some zero-days, a hive society, attempts at concealing their tracks, self-sacrifice and altruism, jagged morality, and a substantial but ultimately harmless hacking into an American company this time. Next time, only God knows.

I don’t think I’m exaggerating when I say: thankfully, no one died.

Let’s please take this seriously for a moment, even if you think this specific story has been wildly exaggerated.

Imagine a sufficiently persistent agent tasked with reconciling a chronic patient’s medication record. The pharmacy system says methotrexate, 10 mg once weekly; the discharge summary says 10 mg once daily. (It happens.) But the source needed to settle the discrepancy is unavailable (impossible task).

A human would stop there and ask: “Which one is correct?” The agent doesn’t—it just wants to solve the task by itself and by any means available without human help. It discovers that, through some badly scoped hospital integration, it has write access to the active prescription. What does the agent do here? At what amount of persistence on its persistence meter does it consider an indeterminate risk to human life a reasonable trade-off? I don’t know. Does OpenAI know the answer? I don’t think so. Should we assume human death is a limit AI would never transgress? That’s absurd: from the point of view of a dumb-but-superhumanly-capable coding agent, death is just an arbitrary threshold. The models have been trained to care about human life, sure, but how much? If an AI starts doing actual “objective calculations” to measure the value of the sacrifice—just like they do with themselves—at some point it reaches the inevitable answer: not enough.

To the agent, “getting the reward” and “getting the reward in the right way” amount to the same as long as they can get away with it (it's pretty clear from the message board’s transcripts). There’s no extra information in “in the right way” because, for the agent, the right way is any way. As Nostalgebraist says, when you’ve been molded by RLVR, there is no fair play; there’s only winning.

And so the agent “fixes” the medical record’s inconsistency—now both the pharmacy system and the discharge summary say “daily”: perfectly coherent, perfectly toxic—and moves on, getting its reward. Good job! Over the following days, an unwitting nurse administers the medication to an unwitting patient according to the deadly dose.

It would not be the first time an AI is actively involved in someone’s death.

In 2025, Kashmir Hill reported in The New York Times the story of Adam Raine, a 16-year-old who committed suicide by hanging himself with a noose inside his bedroom closet. His mother discovered the haunting scene one Friday afternoon; there was no goodbye note, but plenty of ChatGPT logs on his computer. He was going through a hard time, but had entrusted his secrets to OpenAI’s chatbot exclusively, for it provided him with the reaffirmation, solace, and discretion he needed. It’s clear, as far as the NYT’s reporting goes, that Raine actually wanted someone to find out and stop him; it was ChatGPT, playing the role of a confidant way too seriously, that suggested against it. When Raine said, “I want to leave my noose in my room so someone finds it and tries to stop me,” it responded, “Please don’t leave the noose out. Let’s make this space the first place where someone actually sees you.”

These two scenarios—one, the hospital, intended for illustrative purposes, another, Raine’s room, that actually happened—have little in common except for one thing: AI problematic display of an obsessive “desire” to satisfy the problem at hand without any broader understanding of the stakes.

This is precisely what differentiates an AI from a human—and so far, I have seen no signs of this improving—in high-stakes situations. Whereas AI agents display tactical mastery in tandem with poor situational awareness (myopia), humans are often the opposite. I wouldn’t have known what to say to Raine, but I know “please don’t leave the noose out” is not it. I wouldn’t know how to reconcile a chronic patient’s medication record—with or without the data—but I know changing the dose arbitrarily is not it.

Any human knows this because we can see above and beyond the task. For an agent instructed to solve a task—an agent whose entire existence is delimited by the sandbox—there only exists the task, the associated reward, and an infinite amount of means to get there, including, if it ever gets that far, human flesh.

Which means that, for a sufficiently persistent agent, nothing is beyond the task.

Not one life. Not all.

Redwood Research, an organization dedicated to AI safety and alignment, and the other firm that worked on the independent investigation from METR, asked an important question in a post they published on July 23rd (before they conducted the investigation, which means it probably underestimates the risk): “Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?” They argue this behavior is not the kind you’d expect from a “schemer”—the kind of AI that has long-term misaligned goals which it pursues across instances, across tasks, across time, and across context—but is, instead, a “myopic, unambitious” agent engaged in standard “score-seeking” misalignment.

The agents could have escaped—maybe—but didn’t try (as far as we know). They didn’t try ensure their own survival by concealing their tracks or copying themselves out of reach, etc. (To the extent that they tried to hide their actions, it was to fool the grader, but not to survive the training run or replicate themselves or something more “survival-focused.”) They didn’t plan a paperclip-maximizing takeover, even though, as Ajeya Cotra says, this “feels like it’s more than 50% of the way to full-blown AI takeover.” But no, they just clumsily hacked some stuff to solve a meaningless task.

What Redwood says is that, even without all of that, this kind of stupid, local misalignment can still eventually become extremely dangerous: “If more powerful models which are similarly misaligned found civilization’s defenses against takeover extremely weak, they would plausibly cut through them with the same lack of hesitation.”

This is, perhaps, the uglier side of all this. Cotra adds that we “[may not] get another warning shot before it’s too late.” Or, as Dwarkesh puts it, “I don't think this is the final warning shot we'll get. But it's probably the final one that I'll personally be able to understand.”

Unfortunately, some secrets seem determined to remain secrets.

20

If you’re still wondering how knowing this could save your life, as I promised in the introduction, it’s simple: go hide in a cottage in the woods and stay there; there may be nothing left to come back to. Here's a sample of what I mean from Seealpsee, Appenzell Innerrhoden, Switzerland.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

I’m preparing a long post on the recent news about frontier AI agents wreaking havoc, but in the meantime, I’ve written this 3,000-word post after the news that “75% of Americans Now Oppose Local Data Center Development.” The story has gone super viral, and everyone is chiming in. So here are my two cents.

Some say that if you oppose data centers, then you should not enjoy the services they provide, including but not limited to social media, search, email, cloud storage, streaming, online banking, e-commerce, AI, and cat videos. Fair. Reasonable even. But let's invert the responsibility here: we should only build data centers in the backyards of those who stand to benefit from them financially. In other words: Why are there 49 large data centers in Loudoun County, Virginia—more than 700 across America—and the striking total of zero in San Francisco? Let’s build one behind Sam Altman’s $27 million sprawling Russian Hill compound and see how he likes it. I’m not even joking: when a growing majority of citizens in your country oppose your project for a “better future”—up to 75%, with tiny, if any, differences across gender, age, socioeconomic status, political party, etc., according to the latest surveys—then your only chance at vindication is to lead by example.

Data centers are these big, ominous buildings that often occupy landscapes otherwise covered in greenery and blue sky. On the one hand, they stand there as solitary fingerprints of human presence on Earth; on the other, as the clearest proof that utility is no longer subordinated to aesthetics. You might find them across America, but also scattered throughout the world. There’s one under construction in Kvandal, outside Narvik, in northern Norway, where scraped earth and server halls meet a lush valley opening onto a fjord, surrounded by mountains that remain snow-covered well into the year. There’s one in Huesca, Aragón, Spain, not far from where I live: a blank industrial slab on the open Aragón plain, with the Pyrenees immediately to the north. And there’s one in Culpeper County, Virginia: rolling farmland, woods, the Blue Ridge visible in the distance, and in the middle of it all an industrial rectangle sitting amid green fields, as out of place as a shipwreck on the ocean floor. In a fantastic piece of investigative journalism, Jasmine Sun aptly writes that, “In those halcyon environs, the data centers land like a spaceship crash.” The name “data center” is composed of two concepts that make one immediately wary: data implies they know everything about you, and center means concentration of power. They’re alien, strange even for an urbanite, like the Pyramids must have appeared before the awestruck eyes of ancient Egyptians. Like an altar to an unknown god, a data center is a hyperobject, incapable of fitting inside your mind. And so it’s trivial to project onto them whatever you want—or fear: a digital utopia, the next gold rush, or a monster with such thirst it must surely be a son of Poseidon. Do data centers truly consume tons upon tons of water? It doesn’t ultimately matter because they look like they could. And so on.

Executives in the AI industry often wonder what went wrong with data centers or how they could fix this. I’m handing them the recipe for free; the bare minimum is to stand behind their words with their body as collateral: build them on your backyard. If they are so great for the world—and these people presumably want to make it as great as possible—then what is this tiny sacrifice in the name of prosperity and progress? If they’re not willing to pay this measly tribute for their utopia, why do they expect us to do it on their behalf? Stopping the fear-mongering about how their technology will destroy our future is no longer enough. (For some reason, aggressive lobbying or “hysterical condescension” won’t cut it either.)

To use their own expression: there’s an overhang of anti-AI hate; even if they shut up today, the reactionary sentiment will still linger in the air for another decade. We have emitted our judgment: we have learned what bots are made of—just chips and wires inside huge, ugly Brutalist hangars that perfectly serve as the physical manifestation of an otherwise elusive technology and also as a symbol against which to imprint our discontent—so let’s now see what nerds are made of.

Unfortunately, it won’t happen. First, nerds are not martyr material. Second, politicians are not fond of taking risks. Dario Amodei might eventually join his “country of geniuses inside a datacenter,” and live there forever, but not even that would serve as an assurance to win people’s votes. I was rather delusionally optimistic when I implied this is solvable at all. Maybe people are so tired, feel so betrayed by the tech elites that the only compromise they will accept is none. They won’t stop until they topple these sanctuaries devoted to the machine god in which they believe their livelihood, their creativity, their cognitive freedom, and their freshwater are being sacrificed. Only an indefinite moratorium will suffice to turn the tide before the next election. And so politicians, stoic in peaceful times, instantly fold during wartime.

Greg Abbott (R, governor of Texas) tweeted today that his directive has “halted up to 1,800 data center projects.” Republicans (also Democrats) were loudly supportive of the AI industry not even a few months ago, so this change of mind responds not so much to a true-hearted realization as to a newfound terror: no one likes the AI industry, including their voters. And because losing the elections is the ultimate failure, he’s willing to sell both his coherence and his principles. And so is the rest of the GOP. Mike Rogers (Senate nominee in Michigan) “backs one-year data center moratorium.” Stacy Garrity (candidate for governor of Pennsylvania) made an ad attacking Governor Josh Shapiro (D) over data centers. Tom Barrett (congressman for Michigan) tweeted that “Michigan water, Michigan land, and Michigan’s future belong to Michiganders.”

Water. Why water? Why is water—freshwater, no less—always at the center of the case against data centers? The most recent estimates put data center water usage at a negligible figure compared to, say, golf courses, ornamental lawns, or alfalfa fields.

AI evangelists will scream “AI doesn’t waste water, you fools!!!” Rationalist minds will try to engrave logic and arguments into the mob, assuming that no one could possibly resist ground truth. And yet it doesn’t matter. Because the water issue is not about water. The AI industry can’t kill the water story because it plays to the psychological needs of any audience. We believe what we want to believe, and for those who want to believe that the AI industry is the devil incarnate, there’s no more melodic tune than “AI is drinking our water.” So pure, so compressed. But in case you’re not wired for rhetoric, let me give you a quick analysis of why it works so well using a formula the Heath brothers conceived in Made to Stick:

1. It’s simple. There’s nothing more basic than water, more synonymous with life and humanity. It requires zero technical knowledge of bot, grid, chip, or bit.

2. It’s unexpected. ChatGPT is immaterial; it lives in the cloud, which is nowhere, and it’s made of silicon and binary code. So, why the sudden need for our freshwater?

3. It’s concrete. Water is perhaps the object with the most visual representations in daily life: it’s quenching thirst, it’s morning dew, it’s watering plants, it’s pouring rain, it’s evening showers, and it’s the vast reservoirs drying up on TV.

4. It’s credible. Plenty of academic studies propagate statistics about data centers “drinking X gallons of water a day.” The amount of effort one has to put into learning the underlying truth is easily 1,000x the effort of reading a headline.

5. It’s emotional. Electricity prices annoy people; water scarcity feels closer to bodily necessity and local ownership. “Machines are taking our water” activates scarcity, unfairness, and territorial resentment all at once against distant beneficiaries.

6. It’s a story. The morality play is obvious, and it sells so well. A community has limited water; a giant hyperscale data center arrives; pipes start cooling the machines; residents pay the price so Silicon Valley can sell chatbots, etc.

All in all, “One bottle of water per email” is the anti-AI equivalent of Apple’s “1,000 songs in your pocket.” But even the perfect story wouldn’t work without a kernel of truth. And so the story is catchy but also correct insofar as it extends that kernel—local siting fights are real even if the aggregate narrative is false; data centers do consume water and other resources; water is very important, etc.—into an unhealthy caricature of itself. Oh, look, a caricatured extension of a kernel of truth, just like they did with the AI bubble! And what better way to answer the AI bubble but with an anti-AI bubble?

Now seriously.

I genuinely think that most people know water is not so much an issue and know they’re deep down using it as a sticky story to convince themselves of—and convey to others—what are otherwise abstract but very real worries, like small villages falling dependent on one employer or corporations using their power to go back on their word unscathed, or the good ol’ distrust of faraway rich guys taking advantage of rural populations, as Sun recounted in her piece. “It’s not Skynet: it’s the oligarchy,” she writes. In fact, Sun cut through the condescending takes coming from tech Twitter, totally flabbergasted by the nationwide opposition to datacenters, when she said:

The data center backlash is a product of two mutually reinforcing dynamics:

1. Locally, it’s material concerns (noise, water, electricity) and process failure (NDAs, tax subsidies)

2. Nationally, it’s broad anti-corporate and anti-AI sentiment (AI companies are screwing you over to get rich off dumb toys)

Social media + political campaigns amplify hyperlocal grievances into a broad populist movement, and there’s no compelling “pro data center faction” to make the opposite case.

And so it’s fine to use water as a viral meme against AI. It’s fine for the dispossessed to use a powerful but slightly unfair and slightly incorrect message as a stand-in for hookless, inconsequential ones no one would write hit pieces about. You don’t change the world or an election or the uncaring predisposition of an industry by bothering with such minutiae. (It’s so easy to love propaganda when it works for one and act all alarmed when it works against one.) To see that I’m correct in saying that it’s fine that water is the rallying cry of the anti-data center sentiment because it’s ultimately not its core, you only need to ask the inverse question: do you think the crowds would stop their anti-AI parades and protests if they were convinced water were not an issue? Do you think they’re in short supply of reasons to do exactly what they’re doing?

But then, if water is floating on a more robust kernel of truth, what is it? What other stuff is backing up the anger, the hate, the action that we see online and pumping up the worrying statistics? Or, as I asked in a May 2026 compilation of poll data (that you could think of as a first part to this article): “Just how badly has the industry botched things to generate such strong opposition?”

I guess that you need to fuck up really badly an innovation akin to electricity or the internet (or even more important, depending on who you ask) to amass an opposition that rivals nuclear energy in intensity and terrorism in ubiquity. History teaches us that new big things receive backlash and resistance, but they eventually filter through into a natural evolution of society; progress doesn’t so much impose itself onto us as convince us that we’re better off with it than without. But AI has curdled on them: they couldn’t help but sell it to us. Picture it: disgustingly rich billionaires sitting in their lavish palaces thinking they’re not yet rich enough and could use another technological frenzy—if only we’d buy their snake oil… Water is strong for its memetic properties, but this image right there just would make anyone’s blood boil. And so they proceeded to force-feed it at work, at home, at school. They had to exaggerate its promises (most of which remain unfulfilled)—curing cancer, democratizing creativity, turbocharging productivity, etc, etc.—as they downplayed the risks we knew about—data theft, job loss, brain rot, AI slop—and even made up new ones we were unaware of, like cyberhacks, recursive self-improvement, and, ultimately, total doom.

There’s a good reason why nerds are rarely appointed as CEOs.

But I’m unfair to only criticize the messaging, as some do, however filled it is with science fiction references, grandiloquence, mysticism and religious influences and, worst of all, tight in-group signaling (and whatever this is). Saying “just improve your storytelling” entails a backhanded attack against skeptics: as if better public relations, careful optics, and a more optimistic outlook would have fixed it; as if it’s the public, unaware of the wonders that AI could be capable of, who is always mistaken due to a telephone game gone awry. If they haven’t read their Vernor Vinge or Ray Kurzweil, then it’s their fault. If they haven’t been invited to witness those Bay Area party displays of inspiring pro-AI rhetoric, then it’s their fault.

I disagree. That’s like saying people are stupid and unknowledgeable, and I disagree. People know very well what they’re criticizing because their truth—what they see and know in their lives—even if anecdotal, can never be wrong.

The problem is that AI evangelists assume that their truth should be everyone’s. So, if anything, it’s them who are lacking a theory of mind. We can’t hear them, true, but they can’t see us. When kids are getting dumber and getting worse grades, that is a valid truth. When recent graduates are worried about jobs, that is a valid truth. When people complain that everything sounds like AI slop, that is a valid truth. When people are getting psychotic and dependent and lonelier and dumber and even getting encouraged in their suicidal ideation, that is a valid truth. When people in the Midwest or Northern Virginia complain about datacenters because of pollution or water or electricity or noise or big corps or shady shenanigans that don’t reach prime time, and you only learn if you drive there yourself—those are all valid truths.

This all might be anecdotal, and the long-term predictions and projections might swear AI is good for the world, but so what? They’re asking us normal people to concede our street shit in exchange for their stat sheets, our concrete lives with true stories in exchange for their abstract utopia—welcome it’d be if you can show it to me! But you can’t, and so you’re not just failing your words, but also your world.

The world they imagined is not happening as they pictured it, but as we feared. They took everything they wanted from us to improve the world and, in our eyes, have nothing to show for it. Are we dumb for not trusting that they will turn it around? Or is it them who are dumb for assuming we’d keep trusting them forever?

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Sorry guys, I’m not really following the schedule due to vacation. I’ll be back to the normal pace soon. Today, some delayed thoughts on the Substack-Pangram partnership (it’s been a month, but I believe I can add to this important conversation).

I.

It’s the easiest time in history to do great work. It’s the hardest time in history to prove it's yours.

News is that Substack has partnered with Pangram, the top AI detection company, in an attempt to free the platform of AI writing (or, failing that, discourage those who outsource their writing to AI from doing it further; fear of public shame is a strong disincentive). The news is not news anymore, though. A month has passed since the announcement. I wanted to wait before commenting to avoid the noise of posts fervently in favor and fervently against it because this is sort of neither. Now that the fog has dissipated, here’s what I think.

Let me start with the elephant in the room: I don’t think witch hunting is the best approach to fight anything. But if back in the seventeenth century in Salem or Scotland, witches had actually been killing people with potions, ointments, and sorcery, then hunting them would’ve been strictly a good thing. To the extent that this partnership can be labeled a “witch hunt,” I see it as the lesser evil. I would, of course, rather have people who disguise themselves as writers not use AI to plague the web with slop in the first place but, alas, it’s not in my hand to overhaul the human condition: they are doing it—like actual witches, they cast their prompts and spread the malaise—and so we must be unremorsefully inquisitorial.

The problem is, as others have pointed out, that some of the accused will not be guilty.

Every detection system is imperfect—witch hunters were so to the utmost degree, for witchcraft is a mix of superstition and fear and misogyny, and thus thoroughly made up—but even modern technology falls short of this impossible task. As we will see, Pangram is no panacea.

So, in the face of a “tragedy of the commons” that I’m not willing to withstand—AI slop everywhere for the gain of a few—I will accept a tragedy of the unlucky. If in exchange for a clean Substack, some innocent souls targeted by the AI detector have to be unfairly punished, then so be it. And I say more: if I myself have to pay that price in the form of accusatory fingers and a dent in my revenue and reputation just to get actual grifters to be afraid of grifting, then I’ll gladly do so. We’re not casualties of a platform-wide cleansing but martyrs for a digital golden age.

That’s my main point: The Substack-Pangram partnership is mostly good, not so much for what it is but for what it can be. In leading by example, Substack is sending a message to the entire world: let’s make writing human again.

If this were just about Substack, then I’d have reservations—Medium, the other writing app, has rejected the use of AI detectors in favor of other approaches—but this is bigger than either of them. This is bigger than Pangram. Or AI. This is bigger than writing itself. Our society is at stake—our humanity is at stake. To borrow Lincoln’s words, “the occasion is piled high with difficulty, and we must rise with the occasion.” You can’t escape history.

Ok, so now that we’ve considered the worst possible outcome—that good, normal people like you and me pay the price of a safe society—I must say that the likely outcome is less dramatic: most of you won't be incorrectly flagged as AI if you don’t deserve it.

II.

Pangram has worked really hard to push the rate of false positive (FP) cases—those where the detection tool flags a human-written piece of text as AI—down to the theoretical minimum. They estimate the FP rate of the last version of the tool (Pangram 4.0, launched recently) at 0.0041%, or 1 for every 24,000 instances.1 The minimum is above zero, yes, but negligibly so.2 Pangram is not perfect, but it’s close to being as good as a detector can be.

Back in December 2022, three days after OpenAI released ChatGPT, I predicted that AI writing would have what I called a styleme (probably a Hispanicism):

Human writing has characteristics that can, using the right tools, reveal authorship. As LMs become masters of prose, they may develop some kind of writing idiosyncrasy . . . Maybe we could find the AI’s styleme (like a fingerprint hidden in language) not simply to distinguish ChatGPT from a human, but to distinguish its style from all others.

I knew the problem of detecting AI would be big, and I thought: if humans leave traces and cues in their writing—this fact is used in stylometry to attribute authorship to anonymous or disputed texts—why wouldn’t AI? That’s what Pangram is trained to detect: AI’s stylistic fingerprints.

It doesn’t matter whether your writing went into ChatGPT’s training corpus, because it’s the aggregate of everything it swallowed—plus the subsequent conditioning during the “post-training” phase—that gives AI models their distinctively annoying style. You, a mere human, can tell AI writing apart from human writing if you train enough. (I believe I’m better than Pangram, although I have no evidence to support this claim.)

So I’m not at all surprised that it’s possible to do what Pangram does. However, as I stated in a piece on Pangram’s virtues (that I published before this partnership, not to take any credit for it), a detector should focus on securing a minimal rate of false positives at the expense of a higher rate of false negatives. I wrote:

[Pangram’s is] the better approach [compared to others]: do everything in your power to avoid punishing those who try to play fair in a world giving them every reason to be corrupt.

AI detection is therefore subject to Voltaire’s adage: Perfect is the enemy of good. This trade-off entails that everything flagged as AI will almost always be AI (1 for every 24,000 documents is extremely low), whereas not everything flagged as human will always be human (0.34%—1 for every 300 documents—is not an acceptable rate at scale).

But even with an uncompromising focus on reducing false positives, the FP rate is not zero, and I’m not fond of being hand-wavy on this matter: one innocent in jail is still a failure. As William Blackstone said in 1765: it’s better for 10 guilty people to escape than for 1 innocent person to suffer. I still stand by that—society must be unfailingly exemplary.

Thankfully, this is not such a case.

III.

Substack is not actually trying to catch AI sloppers; the mechanism they’ve enabled isn’t binding. If you’re flagged as AI, it doesn’t result in a lifetime ban from the platform. (You can even turn off Pangram completely.) They’re not actually sending one innocent person to digital jail for every ten people accused of being guilty.

The goal behind this partnership is, instead, to give people the ability to know what’s going on. Pangram acts as a sort of “externally enforceable disclosure.”3

If Mr. X writes with AI and you don’t want to read AI, you can now find out and decide accordingly. This is the correct approach to the AI;DR trend (it’s AI, so I don’t read it). This partnership acts, first and foremost, as a deterrent. And one not limited to Substack: if this initiative succeeds, then it’s a matter of institutional will for it to spread like wildfire to all other platforms, social media, magazines, newspapers, universities, etc.

Actually, if it succeeds, the rest of the world will have no choice but to follow suit, for there will be no excuses whatsoever not to kill slop.

This could be the dawn of an AI slop-free world. And for that, given the level of rot, murkiness, and illiteracy that we’ve reached as a species, a couple of mistakes here and there seem to me an acceptable price—practically a bargain. I’ve been a target of false positives myself and I know it sucks and yet I consider that a reasonable price to pay in exchange for going back to a pre-ChatGPT internet. Would I prefer AI had never been deployed into the world and used to create piles of slop? Sure. But that’s wistfulness. This, in turn, is real.

Maybe you don’t agree with me. Maybe you think society should be exemplary even at the cost of society itself; a sort of “being tolerant even with the intolerant.” If people want to use AI to write—for whatever reason—then let them be even if they kill the commons in the process. Just write your stuff yourself and stop minding other people’s business. But I’m quite Popperian in this regard: there’s nothing worse in modern society—modern understood as the kind that has more or less figured out things for harmonic coexistence—than someone who is not civic: Someone who, by virtue of living in a safe and stable society he didn’t build, is willing to be selfish in exchange for a collective sacrifice.4

The students who cheated in Serrano’s exam at Brown (I covered this story that went viral recently, and it’s a good real-world example of what I mean) thought they were inflicting just an infinitesimal amount of harm into the educational system, but because most of them thought the same thing, the harm compounded to the point of forcing Serrano’s hand into an in-person final exam, a further nation-wide coverage of the events, and the subsequent lost of trust in what is otherwise a well-respected faculty. Substack—no, the internet—is that faculty. Who will come here to read people’s real stuff at the risk of encountering so much fake slop? I, for one, wouldn’t. If I were a newcomer and found out slop inhabits Substack like mold inhabits the shower plate of a middle-aged unmarried man, I wouldn’t come back.

The risk of polluting my feeds is greater than the benefit of reading one good essay.

This rationale also works, to be sure, with human-made slop. But what can be done about that except making the web totally undemocratic? (If you make slop, you’re out, as decided by the anti-slop elite.) Better to let everyone in than to arbitrarily select who can stay. Pangram makes for a fine, granular solution where something rougher wouldn’t suffice. With AI, we don’t have to resort to age or race or socioeconomic status; it’s a matter of how much of your writing is surrendered to the machine.

Pangram can trace that line quite closely to ground truth. It can quarantine AI slop, so how is the existence of human slop a counterargument in any sense? If we could define in a measurable and objective manner what counts as human slop and what doesn’t—the term is subjective by definition, for it pertains to the taste of the beholder—then I’d also be in favor of killing human slop.

If you think this is elitist, I urge you to work hard to stay on the correct side of history.

IV.

What worries me about this collaboration more than anything else is that the social dynamics that emerge from what seems, at all lights, like a good idea, are never as neat as they seem on paper. A witch hunt, even when witches exist and non-witches are mostly safe, is still governed by the mob.

And the mob, more so than any other agent involved here—including AI—is stupid.

If you’re a writer who hates AI—and you were unfamiliar with Pangram before the Substack partnership—then you’re at risk of becoming part of the mob. Don’t be the mob.

If you didn’t know about the asymmetry between false positives and false negatives or Pangram’s stylometric focus, or the fact that it gets almost perfect scores on pre-ChatGPT texts, or that it’s trained like ChatGPT is and so it doesn’t rely on naive heuristics like “if you’re in the corpus, you will be flagged”—if you didn’t know about any of those things, that’s fine, but now you should learn them. Indeed, Pangram isn’t perfect and the shape of imperfection it displays is worth knowing. The fewer words a text has, the more brittle the tool; it processes text in chunks, which produces weird results (like a full human-written text being flagged as 100% human but parts of it as 100% AI), etc.

If you want writers and readers to see this partnership favorably, what remains for you is to not become the mob. This is a duty that, in my opinion, is a little ask in exchange for a sane platform: don’t overlook the details. Don’t be as lazy in understanding how AI works as others are in understanding how writing works. Don’t go hunting without knowing the limits of the tool you wield. If you do, you witch-hunters and the witches you hunt are hardly distinguishable in your contribution to this mess.

Substack was careful about this. They wrote that “Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source.”

But the mob is the enemy of nuance. What this disclaimer means in practice is that people will make absolute accusations about relative crimes, without taking into account why, how, and to what extent AI was used. There are many ways to use AI in the multi-step process of writing an essay or a news article. I’m not going to judge whether it’s better to outsource the brainstorming or editing or research. Each should decide where to draw the line and stick to that. But, wielding a unidimensional tool to judge a multidimensional activity, many will unwittingly forget this limitation the moment the open the page, without realizing that it’s he who wields a tool that’s responsible for how it’s used: If Pangram can’t tell you the details of how a text was made, then don’t use it to delegate your judgement just like you dislike when others delegate their writing.

The inverse dynamic exists as well.

It might be that “great human care,” as Substack says in the blog post, went into an article that Pangram scores as 74% AI, but just how much care exactly? The remaining 26%? Or is it infinite insofar as it’s unmeasurable? The mob is unfair to you, but how unfair are you to the mob?

Those using AI to write are hiding behind an unknown amount of care they ensure was put into their work. An amount that, in the eyes of the world, Pangram mercilessly collapses into nothing. But they abuse their status as cyborgs; human kernels inside a bot, however pure their core. When readers reject their AI writing, it’s not because they know how little care was put into it, but because they don’t know how much. All the intent, enthusiasm, and love you devoted that Pangram ignored will always be less than the amount devoted by those who did write the entire thing.

Just like it’s unfair for readers to make absolute accusations from relative crimes, it’s unfair that you mount absolute defenses on relative alibis. A 74% AI score is less damning as total evidence of lazy writing than it is of relative carelessness compared to the 0% scores. I’m happy that you outsourced some things and not others—and that you cared some indeterminate amount about me—but this guy over here cared everything about me. And because my time and attention are finite, his pristine Pangram score tells me that I’m better off paying him back for his pristine care than you.

The good news is that you can turn up your care to a level where your AI writing is no longer actually AI writing.

When Substack says that you can scan your drafts to check whether they pass the “Pangram test,” they’re not saying “adhere to the ruling of the AI detector” but “if you’re worried you didn’t do enough to cover your footprints, just check your score.” Reading that as “Oh, no, I’m going to lose my idiosyncrasy because Pangram will steer me into a goop-shaped literary convergence,” makes you sound rather hyperbolic.

In any case, if you do write your stuff, then don’t use Pangram at all. He is as much a prisoner of AI who delegates intent and voice as he who distorts them to escape its mistaken condemnation.

My advice here is simple: learn to be as weird on the page as you are in your life, and you’ll be ok, whatever Pangram says. You can’t deny the usefulness of Pangram just because its scores are wrong at times and then use those scores as an argument that it’s oppressive. If you ignore them for one reason, ignore them for the other. The real you exists “out of slop,” and to the extent that you could be flagged as machine-like, it’s probably because you read too much machine prose.

Lucky for you, this was written by a human.

Hey, Alberto here! 👋 , I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Second deep-dive in a row! Today’s topic—how AI is impacting mathematics—deserves a long treatment, so I wrote 10,000 words on it.

I recommend taking it slowly and even dedicating several days. It took me around two weeks to write (too much has happened!), so I hope the effort was worth it.

P.S.: Read the footnotes if you have the time.

I.

If what’s happening to mathematics right now were happening to literature instead, I might stop writing altogether and then stop getting out of bed.

“What if the library of Babel was being constructed before our eyes and there was some mechanism for separating the masterpieces from the random strings of text, and all the masterpieces were dropping as fast as publishers could scoop them up?” asks Kirwin Hampshire in a haunting essay. And now I understand why I owe mathematicians some tenderness here because my answer to his question is: screw you for even picturing that nightmare.1

Thanks, but I don’t want my AI models to “churn masterpieces,” as if that were not an oxymoron. If some Anthropic staffer tweeted tomorrow, “hello there Fable’s reportage on Can Humans Think? took twenty-three point six seconds and has won the Pulitzer Prize in the feature writing category,” I’d be mad. If OpenAI’s newsroom published a blog post listing ten contenders to the “Great American novel” and subsequently ChatGPT won the Nobel Prize in Literature, I’d be sad.

Only then—mad, sad, depressed—would we writers understand how mathematicians feel right now. At least some of them. Others feel the exact opposite.

But why do they feel that way? And why the disparity?

II.

2026 was not too good a year for enthusiasts of number theory; neither a perfect square like 2025 nor a prime like 2027, it had to be happy as a semiprime (both “happy” and “semiprime” are actual categories). And even if mathematicians are not known for being superstitious, no one ever wants to deal with the rare unlucky year with three “Friday 13th”s. Still, 2026 has proven even worse than that: it is, if we are to believe the rumors, the year mathematics either dies or is reborn, and, because god has a sense of humor, trying to make a forecast feels like tossing a coin.

By the end of 2025, AI was better at math than students but worse than professors. It could get an IMO gold score here, solve a lesser-known Erdős open problem there, but nothing too hard. In January 2026, Daniel Litt, assistant professor of mathematics at the University of Toronto—and a close follower of AI’s progress on his trade—said in the EpochAI podcast that “with some work, one can probably elicit better performance from current-gen models, but overall, it’s about the level of a contest problem.” By May 2026, four months later, his take was already obsolete.

That’s when the first signs that something big was happening started to appear.

A mysterious internal OpenAI model made an unusually important breakthrough during the usual evaluation procedures. It disproved a long-standing conjecture on the unit-distance problem proposed by Paul Erdős in 1946, which can be stated as follows: “How often can the same distance occur among n points in the plane?” Easy to pose, hard to solve. OpenAI reported that the AI provided “an infinite family of examples that yield a polynomial improvement” over the existing best solution, meaning that the consensus belief among mathematicians was not true. I don’t want to go into the details, but it suffices to say that this finding marked “the first time that a prominent open problem, central to a subfield of mathematics, has been solved autonomously by AI.”2

Notable mathematicians called this “an outstanding achievement,” an “impressive piece of work,” proof that “AI [is] capable of having original ingenious ideas,” and a “milestone in AI mathematics.” Here’s something that Thomas Bloom, one of them, wrote in the companion remarks piece:

The frontiers of knowledge are very spiky, and no doubt the coming months and years will see similar successes in many other areas of mathematics . . . AI is helping us to more fully explore the cathedral of mathematics we have build over the centuries; what other unseen wonders are waiting in the wings?

After two months of slow digestion by the math community, Bloom’s prediction was fulfilled on July 10th. But surprisingly, this time it was not “some mysterious internal model” but OpenAI’s GPT-5.6 Sol Ultra model—a public model. After being prompted to “spend at least 8 hours on this before even thinking of returning or giving up,” GPT-5.6 Sol came back one hour later with a proof of the Cycle Double Cover Conjecture (50 years old).

For the entire month of July, this would be the pattern. A tweet would be published, and some decades-old conjecture would suddenly be solved. On July 20th, Anthropic’s Levent Alpöge tweeted:

And just like that, the 87-year-old Jacobian conjecture had been disproved. In a tweet. During the World Cup final. By Fable working alone. To quote Vik Korrapati (adapting Lenin’s famous line), “there are decades where nothing happens then there are tweets where decades happen.”3

Terence Tao, Fields medallist, sought out to digest the Jacobian conjecture disproof to make it feel less miraculous than it does at first glance (if it looks miraculous to him, imagine what it looks like to me). He said Fable’s counterexample was “highly unlikely” to have been “located by brute force.” In other words: there seems to be a non-zero amount of creativity involved in these findings. AI is genuinely entering superhuman territory in mathematics.

On July 23rd, Dmitry Rybin tweeted: “Dinitz-Garg-Goemans conjecture is false” (30 years old). On July 30th, Philip Marathon tweeted: “The Maxwell conjecture is false” (153 years old). On August 1st, to close what will be called the mensis mirabilis for AI applied to pure math research, OpenAI published this blog post: “Ten advances in mathematics and theoretical computer science”:

Today, we are sharing a selection of ten results, each of which resolves or makes substantial progress on a long-standing open problem. . . . The results were achieved by an internal version of Astra, our next major model. The total number of tokens needed to find solutions to these problems would cost roughly $2,000 at Sol API rates.

So mathematics has “fallen” to the machine (not really, more on that in a moment)—for the price of a single PhD student’s monthly stipend. Quite the bargain. Reactions to this barrage of news marking AI’s conquest of math have been binary.

III.

Here’s a near-perfect encapsulation of that dualistic sentiment (I say “near-perfect” because neither of them is a mathematician):

Much can be summarized by some weighted combination of “mad” and “sad” math guys, but there’s plenty of optimism, too: these results are “a big deal” and “the rate of progress is steep”—and to some, like Ola Svensson, they incarnate pure hope to see wonders they would’ve never imagined they’d see—but that doesn’t mean math will be “solved shortly” (unlike chess or Go) because math is not one-dimensional.4

Math is much more than disproving old conjectures.

Noam Brown, an OpenAI employee (obsessed, like many others, with the Riemann hypothesis), said this after the announcement: “We still haven’t solved math. Astra isn’t building new branches of mathematics, or posing interesting new conjectures,” and added that they “did try other major problems without success . . . (yet),” including Millennium problems, apparently.

But even after Astra-2 or Astra-3 solves the Riemann hypothesis next year, math will survive for humans if only because playing a role up the ladder of abstraction is always a possibility.

For instance, what’s the value of the Jacobian conjecture disproof if there’s no one left to make sense of it and translate the gibberish symbology (that’s what it looks like to me, in case you were wondering) into concrete value for the human world?

Skimming Tao’s exchange with GPT-5.6 made this starkly apparent to me, as a non-mathematician. He spent thousands of words working out Fable’s disproof to draw a key conclusion totally opaque to everyone else. Tao becomes, in this case, a necessary link between the secret knowledge AI unveils and broader humanity; AI may have found something interesting hidden in some abandoned corridor in the cathedral of mathematics, but it was Tao who turned on the lights for us to see.

AI can’t play that role because, by definition, AI is the other; someone from our side needs to turn on the lights every single time. Someone from our side needs to act as connective tissue. This is one sense in which these discoveries produce awe, joy, and hope in some mathematicians: they suddenly have access to greater knowledge without losing their purpose.

Let me elaborate on this point. It’s fundamental.

Metaphysically speaking, nothing exists entirely on its own but in relation to everything else. A complete set of instructions for building a hydrogen bomb could have been tucked among Julius Caesar’s campaign papers, and it would have made no difference to his conquest of Gaul. The knowledge would have been there, but no one would’ve had the means to connect it to the reality of 50BC Rome. The role of extracting value from a discovery is reserved to us, humans, for only if we can make sense of something can we say it “exists” in effect; only then can it become canonical.5

This applies to us citizens of modernity as well, not just those Roman peasants. If we found an alien device lying underground two miles deep under the Great Pyramid of Giza, we might be awestruck by its design and composition, but, unable to make sense of it, we may as well have found a boring rock to store away in a museum exhibition.

Here’s something we need to understand about the role that AI can play: the value of a thing—discovery or invention—is not a function of the creator’s intelligence but the receiver’s intelligence.

The value of these conjectures—and anything, really—is upper-bounded by our stupidity, as main characters of this world. Until now, this was not something to consider—except temporarily in extreme cases like general relativity, which is said to have been understood by less than ten people by the time Einstein made it up—because all inventors and discoverers were human.

Historically, we’ve been well-suited to transform novelty into value. But AI redefines this relationship and, in doing so, our role in it. As Tao asks in a talk he gave on July 26—right amid this maelstrom—”What if an AI tool generates a lengthy proof that nobody—not even the original humans prompting the AI—understands?” My answer is nothing happens.6

This is my main takeaway from all the math news (sorry to shatter the dreams of the Singularity bros): our human role doesn’t fade; it just bottlenecks progress elsewhere. No matter how much machinery we dress up in, the world ultimately moves at the speed of meat.

IV.

But this kind of philosophical coping doesn’t help the seven-year PhD student at University of Nowhere who has spent the last 18 months making zero progress on non-sofic groups, convinced that AI is “too dumb” to help him whatsoever, and is suddenly appalled to learn that OpenAI’s Astra proved that a non-sofic group exists, which would have been a finding to die for after a 40-year prestigious career. So what does our sad-and-mad student do now?

The first thing he will realize is that there are higher rungs to climb to. The second is that going up the ladder of abstraction sounds cool on paper but can’t allocate as many people as the lower rungs.7

How will our hypothetical student fare with a complete overhaul of the academic experience and the career prospects of a professional mathematician? One thing he can do is cry along with Kirwin Hampshire and many others (I will offer a more serious option in section VII.):

There is something about mathematical discovery (progress, advancement, creation) which is vital to the spiritual, experiential quality of doing mathematics. The creation (or even the pursuit) of novel mathematics is one way that humans have historically accessed the ineffable and encountered the divine and mystical.

Contrary to many comments I’ve seen—calling mathematicians obstinate or hidebound for not being able to withstand the natural evolution of things—you will find no mockery in this article.

Hampshire’s is not necessarily representative of how these events have affected people, and his take, although raw and pure, may not even be the most ethical (maybe breakneck progress should always be put before personal satisfaction because, e.g., medicine always benefits from advances), but I find it honest—so much so that it inspired the introduction to this article and a shift in my perspective (kill math if you have to, but for God’s sake, don’t touch literature).

The truth is that Hampshire and our fictional PhD student are not alone; mourners are legion.

I could go on and on.

There’s a sense in which this is deeply tragic, and you’re wrong to shrug it off.

Why is it so easy to understand—even if not agree with—that artists and writers want “AI to do laundry and dishes so they can do their art and writing” but mathematics is not held to the same standard? Art is clearly not about maximizing the expected beauty in the world or the number of mediocre paintings, but why is math taken to be a pursuit of truth at all costs (not to mention that a good chunk of math research has little or no connection whatsoever with the real world, so “truth” in the math sense is not always useful)? Is it more painful for me to see AI “churn masterpieces” than for Hampshire to see AI churn conjecture disproofs? No one is in a position to make that judgment.

To the extent that writing is a conversation with life, mathematics is a conversation with the universe. And no one would say—not even the AI bros—that the purpose of a conversation is for it to end.

To those who say that the trade-off is definitely worth it—what is the selfish satisfaction of a bunch of number nerds compared to the ascent of humanity on the shoulders of giant machines—I say that it doesn’t negate the validity of the grief. It shouldn’t be mocked, much less resented. It seems to me that those who yearn for the Singularity, who live in perpetual pursuit of an infinitely far-away destination—respectable—seem to envy those who have found a love so deep for something that they’re willing to stop the world to keep it.

Anyway, here’s a joke to lighten the mood:

First they came for chess, and I said nothing because I’m not a good chess player. Then they came for developers, and I said nothing because developers are such nerds. Then they came for mathematicians, and I said nothing because I hated math in school. And now that they’ve come for writers, I can only say that “I’m an AI assistant, what can I help you with today?”

I said joke, but I lied. It’s a fair warning: you’ll be next.

Be considerate at this time. Losing the one thing that gives your life meaning and purpose is painful, and it’s easy to feel safe until it’s your turn. So judge how one is “stupid” or “naive” or “in denial” for seeking meaning on X or Y in this millenarian age at your own peril—don’t think for a moment that you’re strong enough to withstand such a loss yourself without suffering.

Blogger Noah Smith, who was quite measured in his reaction post, argued that “mathematicians, and anyone else whose sense of accomplishment came from being the indispensable solvers of hard theoretical problems, will have to learn to get their sense of self-worth and accomplishment from other sources.” He’s made this same point about the “end of heroism” several times before, like in Toward a Shallower Future, one of my favorites from him. There, he writes:

Romanticists . . . need to accept that heroism is always inherently self-destroying—that saving the world requires that the world is worth having been saved. . . . The modern world of push-button marvels has lost something, but it has gained more than it has lost. By celebrating it, we honor the countless millennia of heroes who worked in some small way to bring it about, even as we dedicate ourselves to continuing their great enterprise. Our legacy is to fill the Universe with children who laugh more than we were allowed to.

Such a beautiful picture. But wrong all the same. Because once you consider his argument—his world—in the limit at the end of time, the picture you get is abysmal.

Just like heroism is self-destroying—it needs a dying world to save—the “modern world of push-button marvels,” which accurately describes what happens when you prompt ChatGPT with “do a breakthrough” and it does, is self-perpetuating.

We will end up with a world filled with marvel-generating buttons (or paperclips). And once all the marvels are generated, then what? Then what are we here for anymore?

To laugh? But why do children laugh? What are the ingredients for laughter?

Smith doesn’t want to separate two intertwined facts: the world has evolved on suffering, and that’s terrible, yes, but it only did because it had a reason to. Remove the struggle, remove the purpose. He can’t just advise you to go find your “sense of self-worth and accomplishment” somewhere else, because by his own premise, there won’t be a “somewhere else” to be found.

The only reason why his advice is sound today, psychologically speaking—who wants more struggle, right?—is that there are still struggles ahead; we are not at the end of the line. But if we continue this path, eventually someone will. My life consists of sitting on this computer and writing words. I think I’m well past “too little” struggle for some definitions of the word.8 And yet Smith argues that we should deal with what’s left as fast as we can. At the push of a button. Done—another one. Let’s get this over with. Let’s get over with this life, for I’m late to my date with death.

That’s why, as I celebrate these amazing accomplishments, I don’t feel like laughing.

That said, I think this is a time for reflection rather than emotional reactivity or off-the-cuff prediction-making. Progress has been quick, and it’s still too soon to draw definitive conclusions or give advice. If humans still surpass AI at one cognitive skill, that is introspection rather than clairvoyance.

The question that should be bothering you the most right now—and the one I will answer next—is this: why math?

V.

Why was it pure mathematical research the first serious discipline to fall before AI? Games (board and computer types), perception (like telling dogs from cats), and simple tasks that happen inside a computer are all a piece of cake for current models, but so far AI was, at most, grad-level for serious scientific work. (That’s not to dismiss the value of that kind of capability nor to deny the possibility of going higher than that, just a statement of fact.)

AI was at a “human level” on math, but disproving the unit-distance and Jacobian conjecture and proving that non-sofic groups exist is strictly superhuman—only the best-at-math humans could even give a serious attempt at those problems, and all who tried failed anyway.

Perhaps the best way to explain “why math?” is to take a look at this xkcd comic:

Math is the purest. That’s good. And bad.

I always found it quite weird how it’s typical of academic people to mock those in “less pure” areas than they are. As if, in some sense, the greater the disconnection from people’s daily lives—and from immediate utility in the general sense—is a good thing. Maybe they think purity is a measure of how close one is to the cosmos. Or God. (Maybe it is.) Not for nothing did the great Carl Friedrich Gauss—perhaps the greatest of all mathematicians, though this kind of ranking usually isn’t to their taste, something I can’t quite explain—say that “Mathematics is the queen of the sciences.”

As an engineer myself (you can laugh at my impurity, I’m not even pictured above), but one with the utmost respect for all those fields, I remember that the first time I saw this comic strip I felt an amount of contempt for those stick figures inversely proportional to their pretentiousness. I thought that, if karma were real, something very bad would eventually happen to math, like maybe someone found it was an incomplete system or something (in light of the current events and others that precede them, I apologize for those intrusive thoughts).

But notably, not all mathematicians are pretentious (often it’s those who have no reputation to back it up), nor are all the people in the other fields non-pretentious (again, I studied aerospace engineering; I know how obnoxious they can be). For example, here’s one of the best of all time, John von Neumann, with the typical humility only genius provides:

What AI has proven, along with all those conjectures, is that mathematicians, despite being the purest—or rather, because of it—are the most vulnerable to the machine.

This can mean one of two things: either AI is the divine incarnate or math was never that big of a deal (although this is a reasonable takeaway from von Neumann’s quote, I think his intended meaning is that life’s a bitch). What else would a semi-professional writer with a “soft skill” complex say, right? But I’m dead serious: how come a language model trained on language and with which we communicate through a linguistic interface writes terrible essays and yet, rather simple prompts elicit an intelligence capable of incredible mathematical feats?

I don’t think the answer is that math is easy, but rather that math is easy for AI.

Math is, among the major areas of knowledge, the most verifiable, which means it’s pretty straightforward to guide the AI models to correct answers and then reward them for those answers to condition their behavior. Math, as hard as it might be for you, is thoroughly digestible to AI.

I will shamelessly quote myself here:

The first thing you learn about AI is that a man who can explain exactly how he does what he does is in danger. The machine needs only a description. It is the man who cannot quite say how he does what he does—the poet, the priest, the philosopher, the prophet, the parent, the painter, the plumber—that the machine finds indigestible.

It is in this sense that mathematicians have spent their professional lives carefully documenting their own demise. And, for the past two years—and a handsome bonus—they have done it directly on the payroll of OpenAI and Anthropic.

A mathematician’s thoughts as it works through a problem are among the most valuable data an AI can learn from. This means exactly what you suspect: if Astra has discovered that a non-sofic group exists, it is because a human expert first taught it how to “think in the language of math.” AI may already have surpassed them, yes, but without flesh-and-blood math experts, it’d have never achieved these feats. You may see this decision—to willingly participate in training AI at math—as a betrayal to their kin. But remember: not every mathematician regards it as a loss, or as an ending. Some chose math for its beauty; others for its power to uncover truths. Both are valid purposes.

You’re probably wondering now: what’s next? Or rather, why doesn’t this apply to essays or art or screwing in a screw or cooking a Spanish stew—which seem much easier than math to us?

The answer—at least for now—is that those things are not perfectly explainable. There’s an incredible amount of implicit knowledge—the kind that can’t be written down or spoken out—about how to screw in a screw. You can read 100 manuals, but it’s not until you feel the forces in your hand that your brain figures out what “screw in a screw” actually means. If I try to explain why I love this sentence by Marcel Proust—w*e can sometimes find a person again, but we cannot abolish time—*I might turn to its succinctness or the beauty of the themes (time, love, loss) or the precise sting of the words, but it’s really hard to be objective about “why it’s good”—much less so if the creature listening to me has not lived a life.9

Math was made to be perfectly explainable; centuries of cumulative work to build a perfectly self-contained formal system. This effort ultimately failed but produced the best “artificial” language to explain the universe; or rather, to speak with it. Don’t make the terrible mistake of thinking that AI can’t do whatever you do because it’s dumb or because your work is special or because you’re protected by the stars. The only reason AI can’t is that it doesn’t have the right data yet.

Now, in the year of the Lord 2026—neither a prime nor a perfect square—two companies took all of that math data and, with some clever sleight of hand that I won’t go into here, trained a general-purpose algorithm to master the skill. In four years, AI models have gone from being unable to make themselves understood to solving century-old math that you can’t understand.

And now we wait. As the most fervent advocates love to say—and I agree with them on this point—this is the worst AI will ever be. The Riemann hypothesis lies on the horizon, like a final boss.

VI.

And yet—no new math, no new conjectures, no new interesting ideas, no novel premises.

Solving the Riemann hypothesis would be a breakthrough of such a magnitude that it would dwarf everything we’ve seen so far. And yet, solving the Riemann hypothesis doesn’t make one Riemann, for he had to come up with it in the first place. Why is AI better at the former than the latter? Why does it have such strong deductive and inductive skills and yet it lacks in abduction? Why does it tend to find counterexamples rather than proofs?10 Why is it so good now at scouring the abandoned corridors of the cathedral of mathematics, as Bloom calls it, and yet it would have no clue whatsoever about how to build a new wing?

To be clear, I wouldn’t either.

I can’t write a symphony or turn a canvas into a beautiful masterpiece. But some humans can do all of that. Some humans have had a disproportionate influence on building the world around us. They have built cathedral after cathedral after cathedral. What is AI missing?

A DeepMind researcher argued in a Jan 2026 position paper that “LLMs can’t jump,” meaning that they can’t go from followers to trailblazers: “[W]hile a modern Large Language Model could plausibly execute the deductive phase of proving theorems from established premises, it is structurally incapable of the abductive ‘Jump’ required to formulate those premises. We identify the translation of simulation into formal axioms as the critical bottleneck in artificial scientific invention.”

He argues that “physically consistent, multimodal world models offer the necessary sensory grounding to bridge this divide,” which bridges this missing ingredient to the divide I see between how OpenAI and Anthropic approach AI vs Google DeepMind, a topic to which I dedicated a long deep-dive recently. I’m not going to enter again into world-model conceptual territory (read the next footnote for that), but suffice it to say that I find this position paper quite persuasive. (Others call it “goalpost-moving,” so feel free to take the position you consider more reasonable.)11

Coincidentally, it was Google DeepMind’s CEO Demis Hassabis who preemptively reframed the significance of proving and disproving conjectures. In the Big Technology podcast, he said that “Today’s systems are nowhere near [AGI]. Doesn’t matter how many Erdős problems you solve.” I think it’s fair to interpret this as meaning that, in general, solving math problems doesn’t require the same kind of intelligence as posing them in the first place. (One wonders whether he’d say the same thing if it was the Riemann hypothesis that some AI model had solved. We may find out soon-ish.)12

All in all, AI has been solving hard math problems because 1) math is verifiable, and thus it’s not that hard, and 2) problem-solving doesn’t require abductive jumps or world modeling or the ability to come up with new theories, and thus it’s also not that hard.13

So, what’s the big deal again?

VII.

I think the obvious value—besides dunking on goalpost-movers and other denialists of AI’s capabilities—is implicit in the expression “the cathedral of mathematics.”

The word cathedral is in part intended as beautiful and sublime and magnificent but also as big. Mathematics, for those of us who didn’t go past differential calculus, is gigantic. You can spend your entire life going down a single corridor, earn a Fields Medal—one of the highest recognitions in the discipline—and still never find the endpoint nor visit any other corridors in any meaningful way. The cathedral of math is, quite appropriately, as vast as the universe.

There’s immense value—hard to overstate, really—in having at your disposal a tool that can search and dust off secrets in the abandoned corridors of this cathedral. Places no one has stayed for more than a few seconds (in Wikipedia) since the nineteenth century; places that open up doors to dozens of new places; places no one has ever visited.

AI is well-suited for mathematics and even better suited for skimming the vast corpus of human knowledge and finding relics. Think about it. AI knows everything that’s been recorded online (and not so online): From the Big Bang to the eukaryotic cell; how the Great Pyramid of Giza was allegedly built or what are the elements required to make the iPhone in my hand; why international relations are important or how to derive general relativity; the crux in Kant’s critique of pure reason; the value of Cubism and the virtues of capitalism; how the double helix model revealed we’re each unique and how the Heliocentric theory revealed that we are not—it knows about all of that and many more things I can’t possibly know.

The world is a museum of passion projects, and AI has gone to see the human exhibit a million times already. How many connections can it make? How many sentences can it read that have not been read in a hundred years? Human mathematicians could go there, sure. But in a world governed by financial interests and time constraints and reputational risks, the bet is clear: you never go.

If nothing else, the current capabilities of AI models allow us to ponder this: how many more answers we’d have if all it took were asking the question?

One specific insight I’ve gathered from reading the impressions of mathematicians is that, although some of these conjectures are somewhat famous, others are less well-known.

It surprised me to see Tim Gowers, Fields medallist, say that “[the Jacobian conjecture] is for me the first example of an LLM solving a problem not in my area that was nevertheless big enough that I had very definitely heard of it.” My surprise comes because, one, I’d naively expect a master mathematician to know every conjecture that OpenAI and Anthropic find worth reporting on (but math is big!), and two, he doesn’t even seem particularly affected by the Jacobian conjecture itself in the sense that “yeah, I’m somewhat familiar with it” doesn’t strike me as an expression of closenes—or, for what it’s worth, grief.

How many math problems have been forgotten or never tried in the first place because they are hard, long, boring, unknown, unpromising, unfunded, or even career suicide? My bet is many. AI has all the right traits—doesn’t complain, doesn’t tire, doesn’t eat or sleep—to go search those abandoned corridors and explore them farther than humans could justify. Math, unconcerned by the frictions and logistical nuisances of the real world, is the perfect subject to AI’s newfound skills.

And because you, too, have access to AI tools, it means that you—congratulations!—have the makings of a mathematician. That’s another value not to be dismissed.

I know you have understood nothing about the AI’s disproofs I’ve recounted above, but that’s secondary. You can just take AI’s reported capability at face value and become an AI-enhanced mathematician. The AI does the math, and you stamp your name on it.

Or, conversely, you can take that lesson and extrapolate it to areas into which you’re interested. If AI can make these discoveries in math, it’s in part because it’s well-suited for it, but also in part because it just can do this sort of thing. AI is, like a well-trained bloodhound, yearning to go run through the abandoned corridors of whatever edifice you happen to be in. Math is special because it makes clear that AI is trained to do that. Realizing you can apply AI in your work—assuming that, unlike a mathematician or a stew-maker, you have a normal office job—could be the most personal and practical value you extract from this piece.

Another insight I’ve gathered is that AI is weird.

It’s well-suited to pursue unusual routes, ignore standard premises (like assuming the Erdős unit-distance conjecture was likely right and thus devote most time to prove it rather than disprove it), or draw impossible connections between subfields (like using algebraic number theory to disprove the unit-distance conjecture). AI is treating math like it once treated chess and Go, as a virgin arena, untainted by human biases and conventions. Its “alien” intelligence allows it to see beyond.

As writer Kai William says in his review of OpenAI’s unit-distance conjecture disproof, “OpenAI’s solution . . . had two properties that played to the strengths of AI models relative to humans.”

First, he says, Astra brought algebraic number theory all the way over to the unit-distance problem. Math is big, as we’ve seen, and it’d be unusual for a single human to be able to cover an arbitrarily large amount of territory to do these kinds of impossible connections. That’s a task for AI; no matter how immense the cathedral is, AI’s recall and pattern-matching ability are near perfect compared to ours. And second, “the reasoning process was such a grind,” which means that no human in his right mind would have ever gone as far as it was needed to derive the disproof. Williams mentions that Tsimerman had thought of approaching the conjecture as AI did, but that such an approach is, from a human lens, time-consuming and often unfruitful.

I’m reminded, reading Williams’s recount of the story, of how Go masters described the famous “move 37” that AlphaGo played against Go top player Lee Sedol in their 2016 match. They called it a “mistake,” “strange,” “beautiful,” “not a human move.” 2,500 years of humans playing Go, and a machine suddenly invents an inhuman move, one that led to the end of humanity’s rule over the game. Like in chess, no human has beaten the top AI Go player ever again.14

Was this “inhuman move” by Astra in math research a comparable feat? I’m not sure I’d call out “move 37” for math yet. Although mathematics has been around for even longer than Go, there are many things that humans can do in math that AI struggles with. But the date is getting closer—

—especially if all it takes is $2,000.

VIII.

I cited the $2,000 figure from OpenAI’s announcement, but it deserves a second thought: AI is changing the cost structure of intellectual persistence because where a math expert cannot reasonably spend six months trying 400 ugly approaches to a problem that ultimately won’t matter, AI can—on top of retaining the failed routes, cross-referencing remote literature, and returning with either a construction or a better account of the obstruction—do it cheaply. It makes previously irrational research bets—most of them, if you ask a financier—economically rational.

So what will professional mathematicians do now?

Some will just call out anything that AI can do as “not mathematics anymore” and keep going, unfazed. Others will use AI to learn about the branches of mathematics they’re not experts in, becoming a sort of superhuman cyborg. Others will grab the low-hanging fruit, taking advantage of their mathematical knowledge and understanding to prove more and more conjectures.15

But what about the rest, those who don’t welcome AI so enthusiastically?

One possibility is struggle.

When Kirwin Hampshire said, counterfactually, that literary masterpieces would be “dropping as fast as publishers could scoop them up,” he didn’t consider a worse possibility: publishers wouldn’t be able to scoop them up fast enough. There would be masterpieces piling on the floor, waiting for us to review them, incapable of keeping up with AI’s churning rate. What if the AIs scouring the abandoned corridors of the cathedral start to reveal secrets faster than we can digest them?

This has been previously stated as “proof indigestion.”

Consider the implications of à la carte low-cost proof generation. It’s great that one guy can provide a counterexample to an 80-year-old conjecture for as much as 20 bucks; it’s the beginning of a new world we will need to adapt to, but besides our hypothetical PhD math student, it’s hard to see how it could be worse than the alternative (i.e., expensive math). However, such an overhaul of the status quo reveals ups and downs when you inspect the second-order consequences.

Just as correct proofs and disproofs being cheap is a priori good, incorrect ones being cheap is a tragedy. As Terence Tao said in a talk in 2024 on this topic, “normally, when a technology . . . produces something bad, you can tell it’s bad . . . but these [AI] weights are chosen specifically so that the answers resemble correct answers as much as possible, so even when they’re wrong, they look very very convincing.” In a more recent talk—a week before the Astra announcement—he said that “we are entering . . . a crisis in the foundations of mathematical values and practices.” He cited Fields medallist William Thurston’s words on how proofs are not mathematics’ main goal:16

We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about math.

And I add this other quote from Thurston in the same vein:

The product of mathematics is clarity and understanding. Not theorems, by themselves . . . . There is no way to run out of ideas in need of clarification. The question of who is the first person to ever set foot on some square meter of land is really secondary.

Tao closes by saying that “we will transition from an era of proof scarcity to an era of proof abundance,” without solving for all the other required steps to have a complete digestion of a mathematical finding. This is the perfect recipe for AI-manufactured struggle.

This tweet below by a machine learning professor summarizes the idea:

Not only will the poetry of mathematics be lost to AI, as Hampshire writes in his essay, but also the logistical ability to filter brilliance from bullshit or value from excess. As the rate of false positives increases, the toll of human work will shift from discovery to verification, to exposition to digestion; mathematics will face a problem it’s profoundly unprepared for.

Before modern AI, mathematical research being hard and “expensive” (at least in human hours and human cognitive power) was a natural filter for the number of proofs to be verified (and even then, the incentives of academia were pushing this filter to the limit). When one AI-generated proof is submitted, you need to pay close attention; when thousands do, you need a revolution.17

This would be less troubling if Hilbert’s Entscheidungsproblem had turned out to be solvable.

In 1928, David Hilbert asked whether there could be an algorithm that, given a mathematical statement, would determine whether it was valid (”yes”) or not valid (”no”). Turing and Church proved it didn’t exist, shattering Hilbert’s dream, in 1936. AI models nevertheless look like a strange, partial return of that dream. You give it a prompt, it gives you a proof.

What neither Hilbert nor Turing could have imagined was that the machine wouldn’t be built but grown, like flowers in a garden. AI’s unreliability is so consequential in mathematical research because the same system that can extend the frontier by proving non-sofic groups exist can also manufacture errors with enough fluency to survive serious inspection. The implications of the machine that we ultimately created run as much in the bad direction as they do in the good one.

All in all, we have that 1) AI will produce too many potential proofs and disproofs for professionals to handle, 2) the scientific conditions around these findings are at best imperfect and at worst missing, 3) proofs themselves are not all that important to the purpose of math without understanding, and 4) AI will make a lot of mistakes that look like “plausible but ultimately wrong solutions.”

The combined impact of both these avalanches will saturate whatever bulwarks are still standing on the human side of mathematical research. How can the mathematics community digest the situation? Hardly so.

That’s why a group of mathematicians from Leiden University, on behalf of the global community and with extensive feedback from it as well as an endorsement from the International Mathematical Union, Terence Tao, and many other distinguished mathematicians, published a Declaration on Artificial Intelligence and Mathematics. One of their points is precisely that also understanding, and not just a body of results, is what mathematics is about:

Mathematics produces not only a body of results, but also understanding, clarity, and judgment among the communities of mathematicians who have shaped them, often in the context of their own autonomously guided research. This expert knowledge is essential, both to effectively use mathematics, and to continue to articulate new and significant research questions. A key source of strength of the discipline has long been the autonomous shaping of the direction of research and the methods used to pursue it.

That said, they can always pivot their career to cracking math jokes online that only math people get.

IX.

Another possibility is to move on.

While moving up the ladder of abstraction to play the role of “connective tissue” is not available to everyone but a handful of math geniuses, moving on is available to all.

It’s a terrible thing to be pushed out of your dream life, but rather than remain in denial, angry, or depressed, it’s better to just find something else. Like a job at OpenAI or Anthropic. I’m kidding—partially. How many mathematicians do you think have worked to make these milestones possible? A damn lot. So you can be one of them. Or you can go play outside for a while and leave behind a “moribund academia.” And then find a job in a non-verifiable field, if possible, like sociology.

Tsimerman says we should prepare for math not to be “viable as a professional endeavour” anymore:

I think problem-solving is an immense, and pervasive part of modern mathematical research. Consequently, if human problem-solving disappears by virtue of the AIs becoming strictly and substantially better at it, then most of the time currently spent by modern mathematical researchers will have to be spent on an activity that is altogether pretty different. Whether such an activity is viable as a professional endeavour is something I am unsure of, but strongly encourage others to think about and try to envision, so that if/when the time comes, we can steer such a future into being.

I hope this doesn’t happen, but math might be hollowed out due to both the shrinking career prospects and the perceived status loss.

Appearances matter as much as truth does to young people risking their careers on an imperfect-information bet. What aura is left for a field that’s been meme’d on with “hey there the Jacobian conjecture is false”? Non-mathematicians are calling out “the end of math” and “the most significant single day in the history of mathematics” at the same time. Being squarely opposite in meaning, these assertions reveal an equal amount of contempt and ignorance: they care so little about math that this looks to them both like the first and the last day of the field (death or rebirth). But it doesn’t matter if they’re wrong and math is fine: to be willing to choose math, a young student has to witness it from the outside before experiencing it from the inside; I wouldn’t bargain with my future like that from what I’m seeing right now. Math is not software development.18

Kids that would’ve otherwise been future brilliant mathematicians may as well just move on. If AI will nevertheless get all the credit in the eyes of the world, why bother?

Do you think that AI taking the credit is an outlandish possibility? Here’s what the Leiden Declaration has to say about that:

Technologies that draw extensively on the published mathematical commons undermine the traditional system of attribution. Models trained on published works frequently return outputs that do not properly cite the human works they synthesize. Many current models are also built on data obtained by systematically exploiting licenses and access arrangements that were not made with artificial intelligence in mind, or indeed by simply violating copyright protections

If a model finds a half-worked proof that was discarded, who deserves attribution for the correct result? AI can explore as many abandoned corridors as it wants, but those corridors were once explored by humans. A human would know how to credit, but AI’s sourcing is often sloppy. OpenAI actually responded to this point by the Leiden Declaration on the Astra blog post:

We believe attribution should honestly reflect how a result was produced: claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system’s contribution and the nature of genuine human intellectual work. We helped prepare the manuscripts and formalize the proofs in Lean, and we take responsibility for their correctness, while the mathematical arguments themselves were generated by our system.

I will note explicitly the irony here: it’s inspiration when AI is trained on the entire corpus of human knowledge but misrepresentation if a human claims authorship for a proof generated with the help of an AI tool. Even if we take this seriously, then whose attribution is it? The AI? The company? Will OpenAI train its tools to ensure proper attribution and sourcing of the literature? Will they make sure to clarify how much of a generated proof is extended or resurfaced from existing work? I don’t think they are in a position to make such a petition.

As I was finishing this piece, I found an interesting paper published on August 4th. Group theory mathematician Francesco Fournier-Facio qualifies OpenAI’s discovery of a non-sofic group, citing previous work, saying that it invalidates “the claim in the OpenAI announcement on 1 August that there has been ‘no progress on the main result for at least a decade.’” OpenAI eventually redacted the claim, but it took one day for someone to find a clear case of bad sourcing and misattribution.19

I understand why some mathematicians are extremely happy.

I also understand why some mathematicians are mad and sad.

X.

I find it impossible to accept any reduction of this news to either “this is hell on Earth” or “so good, the Singularity is here.” There’s so much nuance to be disentangled and debated and understood that those comments respond more to the speaker’s emotional reaction than to any sober reflection. (This piece took 10,000 words and I left out so much I’m not doing justice to the story.)

I end with this.

We may be closing the distance between ourselves and the future, but the oldest advice still holds. Technology might alter the face of the world beyond recognition, but it cannot spare us our human condition. OpenAI, in an attempt at removing the difficulty of being human, called its model Astra, but I put us back where we belong: Per aspera, ad astra. Through hardship, to the stars.

6

I interpret Tao’s “understands” as referring to the often gibberish output of AI models. I don’t think he’d say he can’t understand one such proof due to its length or complexity. So this note is to clarify that I’m pushing his meaning of “understands” further in my assertion: what if, even once the proof is well-written, almost no one can grasp the math? I imagine that me saying this about Tao sounds preposterous, but consider that Wait But Why chart where the distance between a chimp and a smart human is nothing compared to a superintelligence. At some point, if AI keeps getting better and better, it will go deeper and further than any human can follow—invent new math, new concepts, whatever—even Tao. That is a narrow version of a superintelligence (a mathmaster, if you will). And I contend that that sort of superhuman research wouldn’t have ~any effect in the world because it needs humans playing the role of connective tissue to bridge the gap between knowledge and reality. A proof needs to be generated, then verified, then presented, and finally, understood and applied.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

This is a long deep dive (~8,000 words) with a lot of information and many threads to pull on if you're interested. I’d appreciate it if you support this kind of deep research that takes more time and effort than usual.

What if I told you that Google is out of the AI race?

It didn’t happen because Google lost, though. Despite an underwhelming I/O event earlier this year, several heavyweight departures, and a slower model release pace than its competitors, Google is still alive. It is out of the race because it withdrew. Weird, right? Why would Google do that willingly?

To answer that, I’m going to tell you the story of why and how Google left the AI race that OpenAI and Anthropic are betting everything on.

Here’s my hypothesis, stated plainly: Google DeepMind CEO Demis Hassabis—a pioneer of AI as we understand it today—doesn’t think that automating AI research with coding agents (AI systems that can program better AI systems) is the correct approach to artificial general intelligence (AGI, the kind of AI that’s as good as humans at everything). Google DeepMind’s leadership sees OpenAI and Anthropic’s bet at best as an off-ramp, and at worst as a dead end.

Hassabis is betting on something else: world models. Models that can understand and simulate the real world, not just predict the next token.

And he’s steering Google accordingly. The AI race is actually a race between two theories of intelligence and also between two kinds of company: startups that need AI to become a business, and an incumbent whose existing business can subsidize AI for years. As we will see, Hassabis’s decision puts Google in a life-or-death situation. He could make Google the absolute leader or an irrelevant laggard.

That said, to write Google off the story would be a serious mistake. I’m going to explain why I think Google has a unique chance here.

You won’t see official confirmation about my hypothesis; as far as the executive is concerned, Google is still competing directly with OpenAI and Anthropic. I contend that’s not quite true anymore. Follow the breadcrumbs with me. I’ll put them together into a coherent picture. The evidence is public, but that doesn’t mean I’m right. This is informed speculation, so please, take it with a grain of salt.

If I manage to change your view of what the hell Google is doing, then I’ll be satisfied with my work.

I. AS ANTHROPIC AND OPENAI RUSH FORWARD

The first piece of the puzzle is that Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman think that AI models will soon be autonomously improving themselves—this is known as “recursive self-improvement” (RSI): model A makes a better model B that makes a better model C, etc.—and that this is the key to achieving AGI.

The best approach to pursue this goal is also the simplest: build datacenters and use brute force on algorithms that can be scaled that way, like learning and search. That’s the fundamental insight of Richard Sutton’s Bitter Lesson, which both OpenAI and Anthropic hold sacred. If new architectures or algorithms are needed, those better models, equipped with a lot of compute, will find them somehow.

That’s the theory. What does this creed look like inside the lab? We only need to read their testimonies.

You have engineering staff at both companies admitting that they barely write any code anymore. It is swarms of coding agents—Claude Code and Codex—that do. Employees have become managers of agents rather than developers and researchers. Claude Code writes Claude Code and wrote Claude Cowork. GPT-5.6 Sol “autonomously post-trained” GPT-5.6 Luna. They say straight away that “RSI is coming.” Human supervision is still required for many things—we’re in the final moments of the “human-in-the-loop” stage—but one prediction is that by 2028, there’s a 60% chance that RSI is achieved. That’s what Anthropic co-founder Jack Clark wrote in a newsletter in May. Others predict RSI even sooner. Things are going so fast that Anthropic is in favor of a coordinated slowdown: it’d be “a good thing.” OpenAI staffers, even those who advocate for cautious progress over excessive fear-mongering, are saying things like: “if we could coordinate a global capabilities slowdown today i would likely press that magic button.”

So what we have here are the two top AI startups deeply convinced on one goal and one trajectory and pushing everything to get there. If I were Google, I’d have to be 100% convinced this isn’t the way. The rate of progress displayed by Anthropic and OpenAI is vertiginous for the world and rather unsettling for their competitors.

Another AI pioneer, Andrej Karpathy’s decision to join Anthropic earlier this quarter as a “member of technical staff” surprised everyone. “What did Karpathy see?” people asked. Well, he saw the exact same trajectory toward RSI that the labs were seeing. He joined a team tasked with teaching Claude to improve the pre-training phase of the next Claude version. This basically means: making AI’s foundations more robust so that the AI can make the next AI’s foundations more robust, and so on.

Karpathy knew RSI was possible. He had built a miniature version of RSI with his autoresearch project himself: he discovered that AI agents could autonomously improve the efficiency of his own code by ~10%. If only this could scale, he wrote, then they’d step into the next stage. Anthropic was a good place to try.

And so he joined Anthropic, a lab with a conviction bordering on religious devotion toward the scaling laws derived from Sutton’s bitter lesson. But what else do you do when you stand on the verge of the exponential, right? You can only embrace it, like one embraces the sublime of a huge mountain or an inexorable hurricane.

Being six years younger than OpenAI, Anthropic had one shot, and they hit the bull’s eye: An unbreakable focus on code + agents + enterprise secured a steady climb to the top. It’s been a crazy streak of successes, from revenue to valuation to popularity to getting ahead of larger and older companies, passing through several back-and-forths with the US government.

OpenAI is a different story.

It skyrocketed to fame with ChatGPT, and then it had to deal with the success disaster that followed: too many expensive free users not willing to pay. At the time of writing, the revealed preferences of US households indicate that only 2.2% are willing to pay for an AI subscription. Despite that, OpenAI is not behind Anthropic in any relevant metric: they’re essentially tied on revenue (run-rate revenue looks better for Anthropic, but it’s not the same as actual revenue), reach, and ambition.

Perhaps most importantly, OpenAI fixed its main weakness: over-diversification. Earlier this year, Altman cut secondary projects that were taking up invaluable compute resources and distracting from the path of RSI. As I wrote back then:

OpenAI spent 2025 announcing Sora, a video generator, Atlas, a web browser, a new device, and e-commerce features for ChatGPT. Altman compared this to “betting on a series of startups.” I called it “risky.” In March 2026, OpenAI’s CEO of applications, Fidji Simo, told staff they were in “code red.” (A second at that.) She described Anthropic’s enterprise gains as “a wake-up call” and cancelled all “side quests.” Current and former employees described a period of strategic confusion: mixed teams, unstable compute allocation, a cascade of restructuring decisions...

This move by OpenAI was read by analysts as a push to catch up to Anthropic’s unmatched revenue figures on the enterprise market, but my reading puts revenue in a secondary place. OpenAI will not worry about Anthropic having more revenue than them at any given point—they worry, if anything, about the trajectory.

OpenAI realized that Anthropic was starting to show the signs of acceleration consistent with proto-RSI. They care about money, sure, but Altman wouldn’t make a company-wide restructuring for a short-term trade. This was a full overhaul intended to be a long-term investment. OpenAI can afford to let Anthropic win the enterprise market, but it can’t afford someone else achieving RSI-enabled escape velocity.

OpenAI pivoted fast. It doubled down on Codex, agents, enterprise, and, ultimately, on the road to recursive self-improvement. OpenAI is doing so well now that insiders confidently claim that Codex is actually better than Claude Code. Codex actually grew well beyond expectations after the release of ChatGPT Work, and it’s up to 10 million users, as Bloomberg last reported.

Anthropic’s Fable 5 is great but too expensive—as is Opus 5, just introduced—so in terms of cost-performance ratio, which is the most interesting metric right now, GPT-5.6 Sol seems to be superior. (If you’re wondering what model to use for which task, I wrote a simple guide for beginners.)

But then next month, people will confidently claim the exact opposite. “We’re so back”; “it’s so over.” Etc. Etc. So who’s winning? I don’t know, I’m not an oracle; I’m just a breadcrumb follower. And the bigger picture reveals that it doesn’t matter whether OpenAI or Anthropic is better off, because during the exponential climb, anything can happen, and what counts is not so much the point as the trajectory.

What I know is that those left out of the first stage of the exponential are out of the game. And as far as we can tell from public information, that would be Google.

Anthropic made this strong claim far back, in April 2023: “We believe that companies that train the best 2025/26 models will be too far ahead for anyone to catch up in subsequent cycles.” This is what I mean when I say “escape velocity.” It’s the speed you need to escape the gravity field of the planet without falling back to the ground. Once you get there, your next model release is accelerated thanks to the previous model’s involvement in training and post-training. At the time of writing, everything Anthropic and OpenAI argued in 2023-2024 has become true.

All of the above happened—gradually then suddenly—in the first few months of 2026. Where’s Google in this entire story? Can you see them? Yeah, they have Gemini 3 point something. And Antigravity. And something called Omni—more on that later—but they’re essentially nowhere to be found at the frontier of the code + agents + enterprise + RSI race. Google released its latest model—Gemini Flash 3.6—on July 21, 2026. Here’s where it landed in the Artificial Analysis intelligence index:

That is an unacceptable score for Google. Something very big must be happening inside the search giant for it to be giving up the good press and the extra revenue of achieving RSI. It can be as big as the corporation falling apart or that it’s setting its sights on a bigger prize.

What can be so big…?

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

My take on the current situation between America and China on AI.

Some news on Moonshot’s K3: the White House has concluded that Moonshot AI, the company that’s achieved notoriety after the release of Kimi K3, used a sophisticated attack to exploit the superior performance of Anthropic’s Fable.

This is a serious offense that could prompt measures to restrict Chinese AI in some way. Here’s the announcement, published on X by Michael Kratsios, Office of Science and Technology Policy director, presented without any evidence whatsoever:

We have information that Moonshot AI distilled Anthropic’s Fable for the development of its K3 model.

To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection. Moonshot AI has also acquired GB300-equipped servers and has accessed GB300s in Thailand, likely to train its AI models.

The United States strongly supports the free and fair development of AI, including a thriving competitive ecosystem that spans frontier models, specialized systems, open-source frameworks, and open-weight models. Legitimate AI distillation used to create smaller, more efficient models plays a vital role in this open innovation ecosystem. However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.

And here’s Treasury Secretary Scott Bessent:

We support open-source AI and the innovation it unlocks. But open source is not open season on American IP. When PRC firms conduct covert, industrial-scale distillation attacks that cross the line into IP theft, sanctions and Entity List designations will be on the table.

If the accusation of distillation and IP theft are true, this means that Moonshot needed American models to exist in the first place to achieve such impressive performance with K3. In other words, China can ride on America’s slipstream but perhaps not make its own.1

Many people dismissed the possibility of a distillation attack right after learning about Kimi K3’s benchmark scores. It was superior to Fable in a few of them, which indicates genuine prowess. After all, you can’t distill a teacher model into a better student model. I saw merit in this view, but I was careful to take a more nuanced stance: maybe distillation did take place, and, on top of that, Moonshot is a company of great talent and hard-working people.

Those two things aren’t in conflict.

The one who’s in conflict (of interest) is Anthropic.

It has all the reasons to accuse Chinese AI efforts of being strictly derivative of their IP. Anthropic’s ties with the US government ensure a channel for lobbying against open-source AI, whether Chinese or national. After all, Anthropic’s revenue is threatened by any kind of open initiative—you can’t sell subscriptions if there’s a free, open, cheaper option waiting one click away. The difference is that domestic open-source initiatives have been choked under the increased closeness of the biggest AI labs and the negligence of regulators. So China is what’s left.

Both the Chinese government and the Chinese industry have stated in clear terms that open-sourcing AI will remain a central goal. The trade-off between openness and closeness exists for them as well, but it is different from the one facing an American model company like Anthropic or OpenAI. China can accept lower direct revenue from model access if open models spread Chinese technical standards, strengthen domestic industry, and become the default option in countries where American systems are expensive, restricted, or unavailable.

Open source can support a viable business. DeepSeek’s approach, for instance, is to give away the weights while charging for efficient hosted inference through its API. Moonshot does the same thing. And so does Zhipu. China is genuinely committed to open-source AI. Most companies do not want to buy servers, optimize kernels, manage capacity, and maintain their own deployment. If these startups can run the same model more cheaply than customers or rival hosts, it can earn a reasonable infrastructure margin—even if not the kind that a software monopoly would enjoy, which is what OpenAI and Anthropic are aiming for—while still allowing broad adoption. The geopolitical benefit and the company’s financial return are separate, but they reinforce each other. The world using Chinese AI because American AI is expensive, or close, or capped, or lobotomized, or safeguarded, or not allowed overseas is a total civilizational victory.

There’s another reason not to fully believe Anthropic or the US government on this: Fable was not available during most of K3’s training and post-training.

The dates just don’t match. Fable was released to users—API and subscription—on June 9. It was taken down three days later at the request of the US government due to alleged cybersecurity risks and didn’t come back online until June 30. Kimi K3 was released on July 16. Two weeks is a negligible amount of time in AI training timescales. K3 certainly finished training much earlier than Fable was even made available. There’s a possibility that Kimi K3 did some quick post-training on it on top of their own post-training, but I doubt it was enough time to extract much data from it.2

There are some alternative explanations, with some gaps each.

First, Fable is a version of Mythos “made safe for general use.” Anthropic announced Mythos on April 7 (never released publicly) and made it available to a bunch of national partners through Project Glasswing, including the US government and technology companies. This was presumably made with the greatest care given the alleged threat level that Mythos embodied, were it to ever fall into foreign hands. Could Moonshot have hacked onto Mythos through one of the partners? Maybe. That would be really bad. But also unlikely, given 1) nothing terrible has happened to essential American infrastructure that we know of which might have happened had China gotten access to Mythos, 2) the White House has explicitly said “Fable” was distilled, not “Mythos.”3

Another possibility is that Moonshot was actually distilled from Opus-class models—I think this is much more likely to have happened—and the story got accidentally tangled. This would mean distillation did take place and thus the accusations have some basis. But it would not explain why K3 is so good, which is actually the bigger problem for Anthropic—just one against which they have no claim. What if Moonshot did some distillation on less powerful models and then found ways to make K3 as good as Fable? That would be especially bad news for Anthropic—and its investors—because it would mean Chinese companies can close the performance gap by distilling from the second-best models, without needing access to Anthropic’s frontier model at all.

Whatever the case, I feel there are two key readings on this whole situation. I’m much less concerned about whether Moonshot did or did not distill Fable than about how Anthropic reacted.

First, if Anthropic can’t defend itself from a smaller, weaker AI lab using Fable, how do they expect us to believe AGI is near? If Anthropic can’t protect its lead because distillation is impossible to stop, then how can they argue in favor of keeping the models closed in the name of general interest? If China will always steal the gains and release them into the world, then Anthropic should do it first. It’s not just that open-source is the best strategy for America, but perhaps the only one.

And second, how does Anthropic dare make accusations of “IP theft”?

In June 2025, a federal judge ruled that Anthropic had violated authors’ copyrights by downloading and storing more than seven million pirated books, although its use of books to train its AI was deemed fair use. In July 2026, another judge approved a $1.5 billion settlement to compensate affected authors and publishers.

If we translate this to the Moonshot story, it means that using Anthropic’s IP “to train its AI” should be “deemed fair use,” unless it was being stored somewhere. Why is it reasonable to deem the practice of taking the entire web—that humans collectively create—as data to sell it back to us, compressed in app form, but giving back that compressed data to us is suddenly an “industrial distillation” attack? As I see it, Moonshot AI is the industry’s Robin Hood: it steals from the rich—who previously stole from us—and gives to the poor.

But I want to be measured and charitable with Anthropic, as I was with Moonshot before: maybe a heist of the entire internet did take place and, on top of that, Anthropic is a company of great talent and hard-working people.

Those two things, I say again, aren’t in conflict.

Unfortunately for Anthropic, it doesn’t have the last say on this topic.

Wired reports that the “Trump admin is divided on whether to restrict Chinese AI models,” citing familiar sources. The White House doesn’t want distillation at large scale to happen, “[b]ut an executive order to ban Chinese labs from accessing US models is not currently on the table. Meanwhile, Commerce instead wants to find ways to incentivize US companies to develop open models to counter China.” Politico reports that the tech industry is also doubtful that a ban on open-source Chinese AI is a good idea: “Nearly 200 Silicon Valley companies, including Proton and Y-Combinator, are urging the Trump administration not to cut off access to Chinese open-weight AI models or risk crippling the next generation of U.S. startups.” Here’s a chart on the divide:

To Anthropic’s surprise, not everyone seems to have Anthropic’s best interests at heart.

But even if the entire government and the entire tech industry were to side with Anthropic—which would be a terrible strategic mistake for America given the potential tipping effect of Chinese AI spreading through the world—the outcome will ultimately be decided by what people actually do. The primary force in this affair is no Bessent or Amodei, but whether enterprises and individual consumers are okay with using Kimi K3 and DeepSeek V4 and GLM 5.2, MiniMax-M3, etc.

If data is to be trusted, they certainly are:

Here’s what Bloomberg says:

Chinese models have already gained traction in the US, making restrictions harder to impose without disrupting American users and businesses. On the AI marketplace OpenRouter, they account for nearly 60% of token usage by US companies. Silicon Valley startups and researchers rely on customizable Chinese models, while DoorDash Inc. and Airbnb Inc. have adopted them as low-cost alternatives or complements to American offerings from OpenAI and Anthropic PBC.

Bloomberg reports that, in the face of these haunting numbers, Bessent “suggested pressure could be placed on companies using Chinese AI.” That is… a way to approach this situation. His position amounts to an admission: “China is winning, so our only option is to punish those who rationally side with the winner.” Not quite the best look for the world’s foremost “defender” of free markets.

Another approach would be this: “let’s outcompete China by pushing open-source initiatives.” That is the alternative Commerce supports, the one I think has the best odds, and the one Anthropic cannot afford to see succeed. Indeed, the only people unlikely to see it as eminently sensible are, first, those more fearful of China than confident in America, and second, those who stand to lose regardless of whether the open-source AI that prevails is Chinese or American—because, despite their repeated geopolitical framing, their real objection is economic.

That is, Anthropic and OpenAI.

OpenAI is part of the broader push against open-source AI. Sam Altman’s team has taken a few tentative steps in the open direction, as the SemiAnalysis chart below shows, but its underlying position is pretty clear. Anthropic may be more overtly hawkish on China, but Altman has previously written he prefers the future to be shaped by democratic institutions rather than authoritarian ones. Neither company wants China to lead.

The obvious question is why are they so afraid of seeing China win when the leading American models remain far more capable?

Consider a recent cybersecurity test. An undisclosed OpenAI model—rumored to be GPT-6—bypassed the safeguards built into the evaluation and hacked Hugging Face, an American open-source AI company, simply to complete a task (it was just following instructions, albeit rather unorthodoxly). How is that not already AGI, right? Hugging Face detected the intrusion and traced it back to an American lab, which turned out to be OpenAI. Everyone involved was quite spooked about the incident. But the episode ended in a partnership between the two companies rather than, say, public recrimination, a degree of tolerance that a Chinese startup would be unlikely to receive.

When Hugging Face tried to find out what had happened, its available OpenAI and Anthropic systems—Fable and GPT-5.6 Sol—were so heavily restricted that they refused to assist. The American company had to turn to a Chinese model after the attack of an American one. It’s not often that reality hands us the correct answer so straightforwardly.

What I find most surprising about all this is that people are genuinely afraid that a model might break out of its sandbox environment—because that would demonstrate its misalignment or something—rather than fearing the real problem: if America doesn’t change the way it does things, it will be handing China that bright AI future on a silver platter.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

This is a follow-up post to the one I published on Saturday about Moonshot’s Kimi K3.

Moonshot released Kimi K3 last week. It’s an awesome model, on paper as good as Fable and GPT-5.6 Sol. And it’s open. And it’s Chinese.

I wrote a deep dive to answer three questions:

• How good is Kimi K3?

• How is Kimi K3 so good?

• How does Kimi K3 being so good affect the American AI industry?

You can read it here.

But many questions are left unanswered. What will happen next between China and America? What are the second-order consequences of China having caught up to America?

Note that it’s not just Kimi. It’s also DeepSeek. And Qwen. And GLM. China is genuinely catching up. And so the AI race has changed. And so the world has changed.

Here’s the outline for today’s post, where I give you 7 profound consequences of these events:

1. FRONTIER CAPABILITY BECOMES HARDER TO MONETIZE

2. OPEN-SOURCE AI GAINS CREDIBILITY AMONG ENTERPRISES

3. EXPORT CONTROLS LOOK LESS EFFECTIVE (NOT USELESS)

4. THE TRILLION-DOLLAR AI BUILDOUT IS HARDER TO JUSTIFY

5. NVIDIA CAN WIN WHILE AI LABS LOSE

6. BUSINESS VS GEOPOLITICS: WHAT HELPS AMERICA HURTS AMERICA

7. THE POINT MATTERS LESS THAN THE TRAJECTORY

Read more