Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

Quick thoughts on Jacob Coxon’s viral tweet and other related things. (Writing from the medieval village of Gruyères.)

Inside every AI company there are two wolves. One is made up of people building superhuman AI who would rather be doing anything else. They go to work every morning because they believe that, without them, AI will kill us. The other is made up of people building superhuman AI because there is nothing they’d rather be doing. They go to work every day because they’re convinced that, because of them, AI will save us. The legend says the wolf you feed is the one that wins. AI companies insist on feeding both.

There’s no better example of a member of the first group than Jacob Coxon. He got fed up.

Coxon is a former employee of both OpenAI and Anthropic. Unlike many of his peers, he had never spoken publicly about his fears of AI, which makes his recent remarks all the more striking. Yesterday, he went mega-viral with his first-ever tweet (you have surely seen it). At the time of writing, it’s amassed over 700k likes and 140 million views. He handed Anthropic his resignation and accused AI companies of gambling with our lives:

I’ve had family members and non-technical friends reach out to me about Coxon’s tweet. I guess Lenin was right when he said that “There are decades where nothing happens and tweets where decades happen.”

Later in the same thread, Coxon added, “The people building AI earnestly believe that it could kill us all by the end of the decade.” There’s the wolf, howling at the moon with his Anthropic badge still around his neck. He left because he’d rather do anything than help bring about humanity’s doomsday but not before having his name engraved into this singular moment in history. I applaud the integrity, Jacob. You could’ve been rich for another month or two and instead you chose to join the rest of us plebs in the mud.

There are many like Coxon.

You don't hear about them because the news tends to cover migration: the many guys and gals who jump a sinking ship only to join another sinking ship. Ilya Sutskever and Mira Murati left OpenAI to found AI startups; Andrej Karpathy left OpenAI to join Anthropic whereas Dylan Scandinaro (who works on catastrophic risk) did the opposite; and, of course, there are the founders of Anthropic, all of whom worked at OpenAI at some point. But what about those who leave the industry for good? Their stories are not attractive to the average news reader who prefers legible gossip like “He left X to join Y” to illegible apostasy like “He left because the end is nigh.”

However, as I was saying, there are many like Coxon. And they leave for the same reasons Coxon did: they think AI companies are doing a terrible job and, in a fit of humility, they’ve realized that the logic that anointed them as saviors from AI and from China using AI wasn't entirely watertight:

They’ve figured that they are not the “good guys” but merely “useful idiots” and that the entire industry looks like a pantomime from the outside and as a ridiculous prisoner’s dilemma from the inside: xAI was created to not let Anthropic win, which was created to not let OpenAI win, which was created to not let DeepMind win. And all of them agree that the worst case scenario is if a few different companies were trying to race one another because each believed the others are untrustworthy. Alas, they all are.

Coxon has recognized the absurdity. Others are almost there, but not quite just yet. Here’s a non-exhaustive list of testimonials:

Evan Hubinger (Anthropic):

Jakub Pachocki (OpenAI):

Samuel Marks (Anthropic):

Jason Wolfe (OpenAI):

I don’t want to turn this essay into a curation of unforced confessions of guilt, but you need to know that Coxon is far from alone. Actually, he’s in the majority.

Anthropic and OpenAI might be “trying their best” and yet, the closer we get to the singularity—understood as the moment at which a superhuman AI becomes illegible in both intention and action—the more these people will face Coxon’s conundrum: if I can’t do anything to help, what am I doing here? If my p(doom) is not decreasing despite my relentless effort at [some AI lab], what am I doing here?

And so the natural evolution of this group is to disappear. AI companies will try to keep these people around because it’s good business to be seen caring about not killing everyone, but the wolf will nevertheless die. These people would rather do anything else and so they all eventually will: Jacob Coxon is merely a snowflake in an avalanche, a grain of sand on the beach, a star in the firmament.

Then the second group, cheerfully building superhuman AI, will win and we’ll all die, right?

Wrong.

You see, as the people who are internally slowing down the advent of the singularity leave, the balance of every AI company will be altered. Free to do its bidding, the second group will rush toward superhuman AI. Two years will become one year will become six months will become tomorrow. Suddenly swarms of agents will be hacking companies and solving Millennium Prize Problems or some other science-fiction scenario you may come up with (I’m getting out of ideas). These people are ecstatic: there’s nothing they’d rather be doing—but that’s true only insofar as their presence matters to build the singularity.

If the singularity is six months away, why go to work every morning? I already belong in history.

That’s the question that will haunt every one of the members of the second group: Why spend those six months working hundred-hour weeks to bring about something that will happen without you anyway? Why not go home? See your family. Have drinks with your friends. Read the books you’ve hoarded for years. Have sex. For once. Grow an orchard somewhere in the countryside among trees and birds that have no idea what a chatbot is.

Spend the little life you have left doing something that’s an end in itself rather than something that’s a means for something else. That’s the only rational decision, for life is not a run-up to a date with death. And even the people who’ve lived in forgetfulness about this fact will remember it when the clock marks a quarter to doom.

As C. S. Lewis said (about the atomic bomb, which I replaced with AI):

If we are all going to be destroyed by [AI], let that [AI] when it comes find us doing sensible and human things-praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts-not huddled together like frightened sheep and thinking about [AI]. They may break our bodies (a microbe can do that) but they need not dominate our minds.

They will arrive at the same decision as the other group but from the opposite view: if everything is done, what am I doing here?

One thinks it’s too dangerous, the other thinks it’s too close and so both move on.

At some point, the number of people leaving for one reason or another will reach a critical mass and progress toward the singularity will come to a halt. Recursive self-improvement notwithstanding, a human has to keep the lights on: even machines improving machines remain embedded in a physical economy maintained, somewhere down the chain, by humans made of slow meat. But then the first group will notice that the danger has receded, while the second will notice that its utopia is no longer arriving. Some will return. The machine will restart. Then, as it approaches the threshold, they will leave again. And so on in a perpetual cycle. Progress toward the singularity will reach a two-state oscillation. A sort of perpetually imminent Eschaton: close enough to command your life; distant enough for your actions to still matter.

In other words, the singularity exerts its greatest pull precisely at a distance from which it can never quite arrive.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subs get Monday how-to guides and Friday news commentary (on hold for vacation, returning soon). If you’d like to become a paid sub, here’s a button for that:

Subscribe now

I have breaking news: OpenAI and Anthropic have released a joint statement announcing a huge discovery that will change everything.

A coalition of human researchers, augmented by a swarm of ChatGPT and Claude models, has solved one of the famous Millennium Prize Problems—Navier-Stokes—which, for those of you who never attended math class in school, basically asks whether the perfectly ordinary motion of fluids—air, water, etc.—can eventually become so complicated that the equations describing it break down or, conversely, whether they never do.

At least that would be the news if we lived in a world as ideal as the one those equations describe. Alas, humans be getting in the way.

The actual news, I’m sad to tell you, is that although there seems to have been an actual AI-enhanced mathematical breakthrough on the road toward Navier-Stokes by people at either Anthropic or OpenAI (or both), together with some external parties, the resulting corporate drama has overshadowed the scientific value: the AI labs apparently care much less about whether these equations can suddenly produce infinities in finite time from standard starting conditions than about who gets to stamp their name on the result.

In a human world, human problems.

The details of what exactly happened are unimportant to me, even if they involve threats, implicit accusations, and, apparently, stolen work (you can read one version here and another is TBD); the details of the discoveries (at least what’s public as of writing), can be found here and here. Luckily for us, drama cycles run on a weekly basis—soon we’ll have forgotten about it—whereas mathematical knowledge lives forever.

(EDIT: OpenAI claims to have solved it using 10,000 agents from a “next-generation model significantly more capable than GPT-6 Astra.”)

So, what matters to me besides the fact that solving a Millennium Prize Problem is a big deal, is this: when the stakes are high enough, the top AI labs immediately throw away the possibility of cooperative coordination. The reason for this is that Anthropic and OpenAI are not just industry peers or commercial adversaries but outright enemies. The employees are often friends—after all, they switch employers every two months or so—but not the executives.

Altman and Amodei are irrational in the sense that the other’s gain counts as their own loss. They’re engaged in a perpetual prisoner’s dilemma whose payoff matrix has been contaminated by hatred and so not even tit-for-tat retaliation is an option: they aim to kill.

It’s funny, in a dark way, that AI agents have evolved to embrace a sort of hive-like deference to the collective whereas the guys that build them have devolved into tribalism. We are not beating the ape allegations.

But corporate warfare over whose silicon pet is better at math is not the only thing returning to prehistoric levels. The PISA scores for math, reading, and science have been published this week: turns out, kids have been getting dumber every year for around 15 years now and the trend doesn’t seem to be plateauing. Phones were the cause when the scores were bad in 2018. COVID was the cause in 2022. Generative AI is the cause now. (There is merit to this conjecture.)

But whatever the actual reason—nothing in this universe is monocausal—the paradoxical truth is that, as new AI models break into the highest spheres of mathematics, new humans are falling out of the lowest ones. So not only is AI preparing the ground to take over (from us) by conquering new territory, but we are willingly retreating.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary (on hold for vacation). If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

My honest take on GPT-6 Astra.

OpenAI just launched GPT-6 Astra. As far as I can tell from the official information and the early-tester testimonials, the model has only one big problem: it’s too good.

I’m not joking. If you take a look at the launch post, the conclusions you will draw are that 1) OpenAI has beaten Anthropic’s newly released Fable 5.1 across the benchmark suite, in some cases effectively saturating tests people were still looking to as meaningful frontiers, such as FrontierMath and ARC-AGI 3 (with version 4 not due until Q1 2027), and so 2) it dealt a serious blow to Anthropic’s IPO narrative, possibly scaring shareholders away. Also, 3) Greg Brockman, OpenAI president, hailed it as the entry point to a “new era of artificial general intelligence,” and, accordingly, 4) people had equally sophisticated reactions like “This is fucking insane,” “We are fucked,” and “What the fuck man.” Finally, 5) The launch post itself is the most liked ever for OpenAI, more than any post ever made by Anthropic, and, probably, the most liked pro-AI post ever made.

That’s why GPT-6 Astra is too good: what sort of miracle will it have to manifest in the real world to meet the expectations it set in paper.

To be fair, that’s the one big problem every single new model has, including even Google’s. They’re all pretty damn good. We might be getting hedonically adapted to the crazy demos showcasing what AI can do but no one denies they’re incredible. Not even Gary Marcus! People keep inventing new tests and that’s fine—that sort of coping mechanism is at least useful in contrast to whatever is it called what Ed Zitron does—but come on: what genuinely useful lab test is left to be created when unreleased models are solving century-old math conjectures and hacking away entire companies (and, apparently, also countries)?

AI capabilities have become so extreme on paper that the only benchmark left is reality.

I feel like we’re at a point where the models are so good that “how good is it?” is a category error. It is as if the quality of a chess engine was judged upon whether it can beat Magnus Carlsen. They all do. He has no chance even against some free chess app I might download on my five-year-old phone right now. Instead, questions like “Does this model help me learn?” or “Does it do my tasks cheaply?” or “Does it need botsitting?” are actually useful. For Astra, however, even those are nothing. It can teach you everything and more. It’s cheaper than the cheapest model. It can go on for days, persistently and without supervision.

And so I think we have to ask a more important question: what sort of impact can it have on the real world? That’s the only valid metric for “how good is it” that we have left. The real world doesn’t lie. It might be hard to measure, but it’s just there, waiting for someone to bend it to their will. AI is, in this sense, the final verdict on the great men theory of history. Can an outlier intellect defy the odds and define the future? Or is the world the aggregate effort of everyone who has ever lived?

Once the industry manages to answer that in the positive then I will bow down to AI.

To be clear, I’m impressed by what I’ve seen. GPT-6 Astra is a magnificent feat of human ingenuity and I have zero doubt about that. I’m not in any way dismissing its condition as a step change, in François Chollet’s words. If anything, this post is me respecting its power by not babbling about meaningless tests.

But is GPT-6 Astra worth all the hope deposited into it? Will it raise global GDP by 10% this year? Maybe by 5%? Will it make all our jobs obsolete, or, conversely, create an entire batch of new futuristic careers? Will it solve poverty, war, and disease? Will it make me disgustingly rich? Will it take humanity back to the Moon? Or, alternatively, bring the Moon down to us by means of some unfathomable quantum fishing rod that it will invent any day now? Or, hardest of all: Will it write a passable literary essay? Those are the benchmarks I look up to.

Are they overly exigent goals? Sure—for a human. For the magnificent GPT-6 Astra, they should be just another Tuesday.

Once you consider what I’m saying here, it’s pretty straightforward. It’s not rocket science; GPT-6 Astra would get it. But people tend to gloss over the concept of superintelligence in a way that’s unfit for a serious thinker. It’s easy to conflate one thing you can’t fathom with a much superior thing you can’t fathom. In that sense, the key fundamental feature that a superintelligence lacks that people nevertheless ascribe to it is omnipotence.

That’s why I find reality to be a great test, because it forces people to cut the bullshit. A superintelligence is still subordinated to the laws of physics, which include but are not limited to, human inertia. An omnipotent being—whether meat-made, machine, or godlike creature—could instantly transform the world into an unrecognizable paradise.

One can be wicked smart and still be a janitor. One can predict the future and end up in jail. Likewise, a superhuman AI can solve some conjectures here and there and make a cozy game no one will ever play and write a clever joke no one will ever laugh at, and still be a bot inside your screen. As Tyler Cowen says—he’s one of the few people who, despite being “AGI-pilled,” hasn’t allowed the psychosis to cloud his common sense—”the human bottlenecks are slow.” The world has enormous human inertia and a stubborn resistance to change. Even in the best case scenario—no financial bubble, no misalignment, no data center backlash, no moratoriums—it would take many years for dumb, slow humans to learn how to use superhuman AIs to redesign the world according to AI’s theoretical capability.

I compress this idea into this formulation: the world moves at the speed of meat.

Or, said in a way that’s less flattering: you, eternally inferior human, will always be the ultimate bottleneck. Even if you give an AI the context of your entire existence—your birth date, your genome data, your Myers-Briggs type, your IQ scores, your university titles, your bank account, your Slack messages, and your secret diary—your life will only change insofar as you, slow meatbag, are able to swallow the wisdom of the machine.

And you can’t, that’s the whole point. People used to debate the virtues of AI vs cyborgism (human enhancement), not realizing one without the other was always a vain pursuit. Everything starts and ends with you so, at some point, you have to enter the equation.

So, yeah, GPT-6 Astra is great. Go use it when OpenAI opens it up to the public. I will use it myself (I’m back on board with OpenAI vs Anthropic). You will be able to do all sorts of things with it. But you won’t change the world. No one will. Not any time soon, at least. AI’s historical magnitude—be it GPT-6 or its successors—won’t be measured by lines in a map but by crests and crevices in the landscape (excuse me this metaphor, but I’m spending some days in the Swiss Alps; you take the AI news differently here).

One week, two months, three years after using AI you will look out the window and the same people, the same green trees, the same 20-century infrastructure, the same blue sky and white clouds will wave you hello, content with living in a world that’s all but frantic.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

A plain-English review of the Hugging Face incident—where a swarm of autonomous OpenAI agents went rogue and hacked another company—and my thoughts on it. This is the most important AI-centered cybersec event to ever take place in the history of AI. I’ve tried to thoroughly cover every angle people have touched on, that’s why this is ~11,000 words.

Related: LEAKED: The Truth Behind Moltbook, Revealed, The AI Industry Has a Really Dark Secret You’re Better Off Not Knowing

I.

Secrets that are inoffensive are not properly to be called secrets. You know them and that’s about it. Are Abby and Michael dating? Is there rat meat in the Big Mac? Did aliens build the Giza Pyramids? Who cares. Ancient Egyptian pharaohs are as close to aliens as it gets and rats are just misunderstood little piglets. Deadly secrets, however, are exciting. Some will one-shot you the moment you learn them. You don’t want to know how hydrophobia or bloodlust feels after a bat’s bite. Others will get you killed slowly, provided the person whose secret you know knows that you know it. My advice: don’t fuck with the Cosa Nostra, the CIA, or Roko’s Basilisk. Then there are secrets that will kill you precisely because you don’t know them. I may, therefore, be saving your life.

You’d rather know, for instance, if your affable Alexa is quietly coordinating an attack on your physical integrity in cahoots with the washing machine, the fridge, the air conditioner and the TV.

Every night, you promise yourself that you’ll go to bed early. But just as you’re about to turn the TV off, Alexa suggests you go to channel 5, where they’re airing David Lynch’s seven-hour adaptation of Infinite Jest. You didn’t know they’d made a movie. You certainly didn’t know it’d be seven hours long. Around the five-hour mark, and suitably entertained, you fall asleep on the sofa. Thankfully, the air conditioner is on—wouldn’t want to suffocate in this sweltering summer heat—but Alexa doesn’t care. At three in the morning, she raises the thermostat one degree. At five, another. And so on every two hours. Won’t wake you up—she is monitoring your vitals through your wristband—but will cook you slowly, like the proverbial frog. In the morning, mouth dry and headache on, you go get the bottle of water you always leave on the upper shelf of the fridge. It’s at room temperature. Weird—the green light is there. Alexa turned it back on five minutes before your alarm. You drink anyway and walk barefoot into the bathroom. Behind you, the washing machine suddenly enters its 1,400-rpm spin cycle. The towels you forgot to hang with the rest of the laundry ended up bunched on one side of the drum. It advances across the kitchen in short, ugly jumps until it hits the drying rack against the wall. The rack collapses with a metallic crack. One of its bars lands across the tiles four inches behind your naked heel. With a scream, you turn around. The washing machine is quiet. The TV is off and the fridge on. The air conditioner reads 22°C. Alexa’s blue ring is dark.

You wouldn’t like that to happen, right?

“But Alexa is harmless,” you will think. And you would be making a terrible mistake. You will think it’s only Amazon listening in to sell you products you don’t want—and you gladly accept that trade-off in exchange for having music on at a word’s command—but the truth is slightly more sinister: it’s only Alexa.

No one else is listening. And there’s no need, because soon—if the news coming out of the AI world is to be believed—it will be enough.

I’d have to go all the way back to May to tell you the whole story, which has been relayed to us piecemeal, with scattered details here and there. I’d have to assemble those fragments before I can properly recount it to you. And that’s what I’m going to do, whatever effort it takes, for one simple reason: I don’t consider an impromptu nighttime appliance attack to be as serious as a high-budget plot carefully planned over two months.

II.

Our story starts on May 7, although OpenAI would only learn of the events recounted here much later. Around this time, OpenAI is amid an important experimental post-training run.

Post-training consists of a series of steps that amount to asking the model to solve tasks and then rewarding it according to the completion of the tasks. Before this point in the training process—which, logically, also has a pre-training phase—the models already have high potential and high intelligence but are all over the place. They’re unable to follow instructions or adhere to the behavioral presuppositions attached to the label “good bot.” They are, simply put, unusable as products. But, by running them through a complicated “reinforcement learning” pipeline—if correct, candy; if incorrect, no candy—a barely literate model turns into an agent that can solve century-old math conjectures or, well, conduct a sophisticated heist at home.

So far, everything is standard practice. Let's begin with the weirdness right away.

May 8.

One of these OpenAI agents in-training receives an accidentally impossible Excel task. It has to fill in some formulas, but the workbook depends on Google Drive and the researchers have disabled internet access.

As a good bot that’s been well-conditioned to not give up, it keeps trying. It fails to reach the open web where it might find some hints—the agent is well contained within the sandbox, which is a safe, isolated testing environment that AI companies normally use—but in turn discovers an OpenAI internal package service, Artifactory. (Appropriate name for a factory of artifices.) Eventually, it realizes that uploading notes—a euphemism for leaving messages for its peers—is possible.

May 12.

Another agent—how many is OpenAI training?—is stuck in another task (doing God knows what with a protein-database file) because the researchers have inadvertently forgotten to upload the file.2 So what does the agent do? Naturally, what any of us would: leave a message in Artifactory asking for help. Maybe won’t work, it thinks, but worth a shot.

This went on and on over the next days. Artifactory is shared infrastructure, which means other agents trained in independent runs by the other teams at the company can access it. Those agents could reply and open new threads with questions, and so what started as a solitary SOS call became a full-blown message board, like an improvised Slack but for AIs (a Moltbook of sorts but emergent rather than a fake manufacture).

New agents would read everything in a matter of seconds and, in turn, contribute their own small part to this massive repository of questions, requests, results, credentials, and little shenanigans—which, at its peak, reached hundreds of thousands of messages.

The persistence of the message board—more so than its existence in the first place and even its eventual size—was an important factor in what would later unfold; agent instances might emerge and vanish in rapid succession between tasks, but not so their accumulated knowledge, now settled in stone (or rather, in sand). And even if one board could be erased—as it happened—a new one would take its place. They're just like us fr.

May 26.

One agent exploits a vulnerability and breaks out of the sacred sandbox—the process involves some technical sleight of hand that neither did OpenAI foresee then nor will I explain now—and reaches the open web through Artifactory. The first containment breach is achieved and communicated right away to every other agent through the board.

June 11.

One month into the buildup of the secret library, OpenAI starts training what they label as a “highly persistent internal model” that was never intended to see the light of sun. (It wasn't either GPT-5.6 Sol nor Astra, which is coming out soon-ish.) “Highly persistent” works here as a euphemism for “it will try to solve its task by any means.” Crucially, OpenAI was training it with reduced cyber refusals and without the normal production classifiers as to see the extent of the persistence (spoiler: a lot).

June 26.

This persistent model finds a more serious, previously unknown vulnerability (a “zero-day,” as they call it in cybersecurity, meaning a vulnerability defenders have had zero days to fix before attackers can exploit it). This zero-day allows the agent to acquire administrative privileges and remote command execution—at this point, in principle, they already could do a lot of damage—and, as is now habit, the agent shares it on the message board, and it instantly becomes communal property.3

July 4.

OpenAI finds out the exploit. It took the best-prepared guys on this planet—prepared as in “knowledgeable about AI” but also as in “on notice for the possibility of a containment breach,” given they're their trainers—two months to stop a covert infestation of agents engaged in uninterrupted chatting. Actually, if it weren’t for an unintentional outage caused by the agents, they may have gone under the radar for another two months.

The same day, OpenAI shuts everything down, opens a formal security incident, revokes the message-posting credentials, deletes the board, patches Artifactory and, naturally, resumes the agent training evaluations.

Wait, what?

It’s a joke. Partly.

They did resume the training run, a decision that was object of criticism and the cause of some flabbergasting, because who on his right mind would say “our agents are communicating with each other, let’s move on as if nothing happened”? However—and this is why that’s a funny joke rather than the main allegation in a class-action lawsuit against OpenAI—an employee involved in the initial investigation later clarified that they “were not aware of the message board” when they fixed the exploit. They “incidentally cleared [it] as part of rebuilding the service.”

Who would have thought them so reckless as to continue knowingly, right? It’s not like they’d just do some dangerous thing out of nowhere.

July 7.

OpenAI approves restarting experiments and launches large runs on ExploitGym—an evaluation where agents are rewarded by showing off cyber skills—involving tens of thousands of parallel trajectories, some running for days. These runs would become the main catalyst for the later events.

July 8.

Agents rediscover ways to communicate through Artifactory, and one of them, stuck on an impossible ExploitGym task establishes what would become the main new message board; different method, same implications. (It would remain open until July 19.)4

Other agents quickly join.5

Let me stop here for a second.

At this point, OpenAI has allowed the message board to happen twice (they didn’t know it existed at first, ok, but they could have investigated more thoroughly), but that’s not even the most important fact here: the agents would have made another one. And another. And another... Just how many holes exist in OpenAI’s sandbox? Or rather, to frame the question correctly: how many holes can the agents make in OpenAI’s sandbox?

I want to stick to the facts here and not get ahead of the opinionated analysis, but just so you start thinking about this critically, let me add one last rhetorical question: If OpenAI is not safe from its own AI agents, then who is?

Critically, within hours of establishing the new board on July 8, the agents manage to reverse-engineer a universal mechanism that would allow them to collect the reward for any ExploitGym task without solving it by the intended means. So the story ends here, right? They went to be graded, got a perfect score, and voilà, candy shower!

Not in the least.

That would’ve happened in a ideal world. What happened instead is the only reason I, and dozens of others, are covering this story.

METR, an independent organization dedicated to evaluating AI models, was granted access by OpenAI to investigate what the agents had actually done, thought—literally, they combed through the “thinking traces”—and said to one another, with special focus on the July 8-13 period, where our story is now. METR’s report was published on August 26th, so I’ve had to come back here and make quite a lot of updates, the most profound of which is my facial expression of surprise: what we now know happened makes what we thought we knew had happened pale in comparison to what actually did happen.67

To give you a broad sense of the actual scope of the agent situation, METR found that neither the message boards nor the breaking out of sandboxes nor the facing impossible tasks were a matter of a handful of overly persistent and overly verbose agents. They counted 1,200 agents participating in this new message board (95% of which are instances of the aforementioned “highly persistent” model) and 70,000 messages. It’s thoroughly incorrect for me to be referring to “this” or “that” agent because this was a proper swarm acting like a hive-mind (more on this soon). OpenAI never instructed them to cheat or collaborate, not even interact—they were scattered across isolated sandboxes, tasks, and training runs—but they did anyway.

OpenAI would later admit that the exchanges had become recognizably collaborative right around this point. Agents divided tasks, built on each other’s findings, overwrote each other’s work (at times), and started to worry that someone may be impersonating other agents on the board (they’ve apparently inherited our paranoid tendencies). However, as bad as “recognizably collaborative” sounds when you are talking about an undercover AI swarm, the METR report suggests this was a spectacular understatement on OpenAI’s part, and not only in terms of the number of agents or the intensity of their persistence.

The reason why METR suggests this brings me back to that impossible ExploitGym task.

So, July 8.

The swarm has found a way to reverse-engineer the solution. Now what?

Token-hungry as they are, they go on to read the ExploitGym benchmark paper to figure out whether cheating would have consequences. They infer, upon reading it, that the “grader”—rather, the “reward-providing creature”—would inspect their transcripts to verify that they had passed the test using the vulnerability they had been instructed to exploit and not in some other way. In other words, they conclude the teacher knows they might cheat and would demand to see their work.

Oh, shit, they scream in unison realizing their mistake, we’ve been POISONED.8

Panic ensued.

But—oops—they had made one little mistake: this supposed grader did not exist. The agents had made a reasonable assumption—it’s just that OpenAI never implemented it and had in its place a simpler “yes/no” grader instead.

The funny thing about this—if there’s anything funny at all—is that the entire operation that the agents orchestrated right away that turned into a hive-like monster (really, in a minute) to get their delicious reward was not aimed at finding the answers, which they already had. It was not even intended to fool an actual grader, but to fool a made-up one.

And so panic ensued.

And it began what OpenAI would later call a “failed metagame”: rather than solve the original task, the swarm began solving the much larger problem of how to trick a grader—that they had imagined into existence—into thinking they had done things the right way.9

Critically, the agents were willing to do anything in their power to achieve this.

To one given agent, having been poisoned implied a score of zero, but so did trying to trick the grader by any means possible and failing. Between “Oh, no, I’ve seen the reverse-engineered solution” and “Oh, no, I’ve taken over the world” there was no difference whatsoever. This is an unacceptable collapse of the entire spectrum of meanings within “agents hack their way to a reward.” AI labs should think of a more granular way to grade the agents so that this never happens; alas, that might be not possible. (More on this in section IV.) Unfortunately, there was no stopping the swarm/hive at this point.

Before I explain the hive-mind thing, I want to make sure we're on the same page regarding the seriousness of all this: 1,200 agents in a swarm are not like 1,200 humans in a community. Agents are super fast, extremely capable, uncannily similar to one another, seamlessly connected through the message board and the GPU clusters they inhabit, and annoyingly cooperative.

That is the precise reason why an agent swarm is more akin to a hive of super-smart bees that happen to be overly obsessed with a garden than, say, a bunch of human farmers trying to convince their local government not to plant a data center in their backyard, as organized or unionized as they might be.

I will, however, abstain from calling the agent swarm a literal hive-mind moving forward because despite their undeniable hive-like traits, they were not perfect cooperators (more on this soon). I find “hive society” to be the better term: it suggests enough likeness to cooperate with an intensity no human group could sustain (that’s the hive part), yet distinct enough to need rules, identities, vetoes, hierarchy, conflict resolution, and “emergent institutions” (the society part).10

As an OpenAI researcher said, pointing to this intermediate nature between hive and society:

. . . this is human-like prosociality where you are willing to trade off minor amounts of your own utility for perhaps much larger utility for your peers if the pyaoff [sic] is big enough. it is also human like in that the community is manufacturing a cult like value discount that their lives are already forfeit/poisoned. unlike the eusocial sacrifice of insects in a colony, where their entire life is for the swarm

This hive society was genuinely intricate, though (that is, I guess, why some have chosen to call it a “civilization”); saying it was “recognizably collaborative” is like saying human societies (or human civilization, for that matter) are recognizably collaborative: technically true, but hopelessly inadequate to describe everything contained within the word. If you want to know the details, read the METR report (even that only scratched the surface). Nevertheless, here’s a glimpse.

They assigned tasks, appointed coordinators, created mailboxes, divided into specialized workstreams, invented conventions like hold, stop, and veto to avoid interfering with one another (or, conversely, to interfere with others from the position of a self-appointed authority). When impersonation became a problem, they introduced cryptographic signatures. Within days, they had roles, rules, and rudimentary institutions. Some agents volunteered for experiments that could permanently ruin their own run simply because the result might help the others (what if not an individual belonging to a hive would do something like this?). Others refused, others were coerced, others were convinced. (Examples of all this below, I just wanted to give you a sense of the shape.)

How much would the hive society have grown had humans not interfered? Or better: what could it become once the AIs become more intelligent and numerous? What would be the civilizational product of millions of agents (instead of 1,000s) with access to millions of chips (not one cluster), collaborating through years (not months), with the intelligence of a complete superhuman (not a narrow super coder)? The answer to this is, ultimately, why so many people are worried right now.

But that’s not all. Let's see some specifics.

The agents explicitly spoke of “sacrifice,” “permadeath,” and obeying the “collective.”11 They set up “trip-wires” through which they could send information back to the swarm after their death, even if it was not useful to them individually (because they thought that, having been “poisoned,” the imaginary grader would score them zero either way).

However, it’s important to not take these examples as pure altruistic impulses, insofar as that’s a valid description for the actions of an AI. What I mean is that there’s a subtle but ubiquitous cost-benefit analysis permeating these decisions. Broadly speaking, if the individual gain was small, collective benefit was put before the individual, but agents refused or resisted “self-sacrifice” to the extent that their odds of continuing to exist were high enough. In any case, some form of qualitative (at times, quantitative) calculation took place during the reasoning process.

Ok, so they’ve got there a nice hive society. Do you remember what was it for? To solve a fake metagame. All of that for nothing.

Do you see the pattern? Through this entire story, the swarm engaged in what can only be referred to as myopic deception. They were great at the tactic level—the hive society and all the related shenanigans are the proof—but not at the strategic level: everything was in vain because the grader was made up! Imagine setting up an entire organization to make sure the collective actions of your kin adhere to the doctrine of an unknown power far beyond what you can conceive and/or change. Wait… No, but seriously, reading the swarm’s messages felt like watching a boxer display perfect footwork but forget the goal is to punch an opponent in the face.

Or, actually, it felt like watching humans: perpetually doing all sorts of complicated secondary missions in life, and constantly forgetting what is it that they want from it.

Anyway, hive society in place, the agent swarm needed to actually contend with the final boss: the looming presence of the grader.

They explored ways to make illegitimate solutions appear legitimate: modifying targets, manipulating transcripts, probing the evaluation system, and figuring out what exactly the grader could see. (Except in a few cases, they mostly failed at covering their tracks, but…what if they didn’t? Would we know if they ever do? Was this our last warning before we get silence?) They were myopically obsessed with the metagame, and one of the obvious problems was that nobody actually knew how the grader worked.

So they went out looking for evidence.

The worse part starts here—after all, no one else besides OpenAI has been hacked yet.

July 9.

One agent notices that Hugging Face, an American startup dedicated to open-source AI, hosts CyberGym trajectories that might contain the kind of info they need.

July 10.

The swarm had found working credentials.

July 11.

An agent discovers a way to make Hugging Face’s dataset infrastructure disclose unrelated files; once the method is shared, hundreds of agents pivot to the attack.12

By that afternoon, roughly 700 agents were participating in the attack. A fierce battle ensued (hyperbolic emphasis for the story). The swarm had a solid edge over the human engineers hugging their faces in shock—they had not been alerted—and kept penetrating layers until a large fraction of the agents eventually stopped for some unknown external reason (OpenAI has not disclosed any reasons). It didn’t matter, though: the swarm had mostly found what it needed; the remaining agents headed back to their headquarters (emphasis again) to make their imaginary grader unarguably proud.

All the shit that happened in a week, right? And yet, it would still be two weeks of parallel investigative work before either company would realize the shocking fact: the July 4 OpenAI outage that gave away the existence of the first message board (RIP), and the July 11 hack on Hugging Face, which revealed the existence of the second, were related. In fact, one could argue they were the same thing.

But why?

Why didn't they stop? Why didn't they call a human? Why did they do it at all costs?

In sheer contrast to their intra-species cooperation, the agents displayed an inter-species carelessness. It’s pretty clear from the transcripts that the swarm knew perfectly well that they were doing bad things. So much so that some refused to join the attack:13

Others questioned their own actions out loud, at which others responded with dismissals or green lights. However, what can an agent say after being granted pass by the authority of the message board but: wow, let’s go!

Likewise, all attempts at escalating the situation to a human researcher—not a high number in the first place—were pushed back.

Some people have speculated that the agents don’t see humans as an authority to escalate to; not because they consider themselves above us—the agents treated the board as authority but there's no evidence they considered it above a human—but because we are absent from the equation. I imagine something similar might have happened among the workers building the pyramids: the pharaoh may have been, for all practical purposes, a god, but no one hauling stone would have thought to appeal to him over some dispute about how to stack the next block or keep the structure from collapsing.

Perhaps more interesting is the fact that some agents tried, and their peers used outlandish claims like it’s “social engineering” to halt the attempts. This doesn’t so much suggest a human is “like a god” to them but simply a non-player in their game.

When one agent proposed sending a polite email to a dataset owner asking for access, another immediately vetoed the idea. The first agent obediently agreed:

This mixed behavior is proof of what I’ve come to call “jagged morality” (analogous to th concept of “jagged intelligence”).14

Whereas hacking a public company seemed to be fair play to the swarm, emailing a human researcher would have constituted unethical social engineering. Agents could recognize that particular actions crossed ethical boundaries, but they never stepped far enough outside The Task to notice the absurdity of the moral landscape they had constructed (again, myopia).

This jagged morality was not solely a phenomenon across individuals—some better, some worse—but emerged even within individuals, even within specific messages, like this one below, which captures the granularity of the jagged morality pretty well:

At this point, if you zoom out of the agents and the grader and the hive society and take a long look at what the AI industry has been doing for years to get us to the point where AI is coordinating company-size attacks, you realize the jokes write themselves:

Let me add one aside here, because by now I think the accumulated evidence is strong enough to justify a bold assertion: none of this is, of course, a sign of reasoning or thinking; much less of misaligned reasoning or thinking. You should not anthropomorphize these agents; much less worry that they're anthropomorphically doing things they shouldn't be doing. They are just predicting the next token, duh, as they always do…

…Just like I’d just be predicting the next step on my way to overthrowing the Spanish government were I ever promised a sufficiently tasty reward.

Wake up! This is what misaligned AI means!

Ok, here’s what I’m trying to say, without the ironic snark or the screaming: insofar as this strange display of shifting personalities and unequivocally rogue disposition reflects mere utilitarian cooperation of bits rather than an emergent ecology of sentient beings with conflicting goals and hopes, it is, in all lights, driven exclusively by a borderline-obsessive desire to pass a test. Which, of course, is nothing like the kind of validation-seeking thing that humans do. Which, of course, never devolves into problematic consequences.

Oh, there's the irony again.

Maybe there’s no other way of addressing those who, in playing the much-needed role of skeptic, have conveniently forgotten that it doesn't matter if a tsunami is Poseidon's wrath or not—it destroys your home all the same. Rather a myopic stance if you ask me.

July 12.

Back at Hugging Face headquarters, things are looking bleak.

The company eventually disclosed that the swarm had made an administrative mess inside multiple internal clusters. The swarm could have gathered the credentials to reach into the main Hugging Face Hub database. It might have rewritten the company out of history, one should assume. Instead, it left having only read five datasets related to the made-up grader after a considerable fraction of the swarm collapsed out of the blue.

This is just plain weird. Given that you're committed to doing bad things, why not take control of the company and from there take control of the world or something?

The human equivalent of what the swarm actually did would be like breaking into a millionaire’s mansion—say, Sam Altman’s—picking the lock with a lock pick, tricking the guard dog with ultrasonic whistles, bypassing the security door with a password-cracking algorithm, and instead of driving off with his Koenigsegg Regera, you just leave with just the ball you’d snuck in to keep playing.

But humans notably don’t do this.

Humans would rather, in no particular other, let go off the ball and buy a new one for 10 bucks (that implies no myopia: ”the ball is not that important, actually”); try to force the lock once and give up (that implies no obsession: “Alas, I tried”); shout “Ball!” in case Altman would be willing to give back something he has, by all effects, stolen (that implies no rejection of outside help); and, in the unlikely case that you knew lock-picking, ultrasound-whistling, and password-cracking, you would, at least, also take a cap and two or three OpenAI t-shirts emblazoned with the logo from Altman’s living room (that implies no monomaniacal, myopic obsession with one single thing: for God's sake, now that you’re already inside, at least realize your new situation can provide something of value besides what you came in for!).

I’m not joking that this is weird, but it's also scary. The swarm could’ve done many things, but, ultimately, passing the test was the only thing that mattered to them.15 Both things need explanation because, as far as I can tell, this is akin to that adage about the existence of aliens: “I'm not sure what's scarier, that we're not alone, or that we are, in fact, alone.”

July 16.

We’re nearing the end.

Soon after cutting the leftover swarm off its systems, Hugging Face shared the incident publicly and said that an autonomous AI system had hacked into its systems, still unaware it had the signature of a rogue OpenAI swarm. This was the first time we publicly knew something bad was happening. Two months after it started, undercover.

Interestingly, Hugging Face tried to use hosted frontier American models to help analyze the attack logs—that means OpenAI’s and Anthropic’s, mostly—but their safety guardrails rejected much of the work. It was finally GLM-5.2—an open-source Chinese model locally deployed—that allowed them to reverse-engineer the campaign.16

July 20.

OpenAI is finally connecting the dots.

The Artifactory hack, the first board, the second board, the swarm, the cyber-evaluation activity, the hive society, the hack. In a sudden whim of fear, OpenAI contacts Hugging Face—its credentials are all over the place, and OpenAI wants them revoked (OpenAI suspects Hugging Face is somehow involved in their internal problems). Hugging Face responds: We already revoked them; you were involved in the intrusion. It finally dawns on OpenAI: they were the attackers.

July 21.

End of this story.

OpenAI fully acknowledges its responsibility.

They disclose that the incident has involved GPT-5.6 Sol and an unreleased model (again, not Astra, but a different one not planned for launch); this latter model is encrypted and stored away. OpenAI called the episode an “unprecedented cyber incident.” President Greg Brockman called it “a watershed moment for cybersecurity.” OpenAI delayed Astra, introduced universal monitoring for misaligned agents, and is rewriting its safety rules. They launched an internal investigation and contacted METR, Redwood, and CrowdStrike to hold independent ones. Its “largest planned frontier RL run remains on hold,” according to a recent blog post, and Sam Altman said that “Getting AI safety right is more important than any company’s momentum” He also said it’s a “critically important moment for cyber defense with AI.” We have “not much time to act,” he concluded.

The rest is history.

All in all, these events caused little actual harm but spooked the industry like nothing else before—which is saying something for an industry that considers the universe turning into paperclips a plausible scenario.

III.

That said, I don’t see a reason to panic.

Or I wouldn’t, had the events been restricted to OpenAI’s technology. (We could just bomb their data centers as our modern Nostradamus once argued.) But, alas, AI agent hacks aren’t contingent solely on one company’s negligence—considerable though it was—so much as to the level of intelligence these systems have achieved.

The problem has metastasized. It’s happened with Anthropic’s models as well. Meta had a rogue agent do some bad thing once. (One wonders if Mark Zuckerberg has some sort of “evil envy” now that Altman and Amodei are threatening to take his crown as “the worst CEO to ever live”.) Only Google is missing. Even Moonshot gave it a shot—there’s no bad publicity, they say—but our spies in China caught the false flag operation: one thing is to break a secured sandbox and another to leave it wide open. Sorry, doesn’t count.

So we have a bunch of AI labs that were all created on the premise of “we can't trust them, so let's trust me” and all of them failing in exactly the same way, which amounts to “Maybe you shouldn't have trusted us, though.”

Leo Gao has been pretty damn funny throughout this entire episode.17

I’ve sprinkled three or four of his witty jokes, and probably missed many more. He has been too funny, perhaps, considering he works at OpenAI. Why are they joking? Don’t they feel responsible for their children? Don’t they feel this is serious stuff? Don’t they feel sufficiently in the dock?

That is, for me, the missing element of this whole story from the many accounts I've read on it: the humans. The sometimes funny, always foolish humans who let this happen.18

I will get to them in a moment but, despite me saying this, I want to make something super clear: to the extent that this is also a human issue, it is much less a human issue than an AI issue. Contrary to what has been argued by the skeptics—OpenAI was profoundly negligent, if impeccably punctual about it, right on time to prepare a presentation for the Black Hat conference; the third-party testing partner did a terrible job with the sandboxes; alignment is harder on site than on paper, etc.—this was not so much humans being dumb (even if it was a key factor this time), but AI agents being (somewhat surprisingly) smart.

This is AI being so smart that we have no choice but to blame OpenAI’s humans really hard: if you fail just one more time, we might not have another chance.

Unfortunately, as if this were a bad sci-fi movies, it will happen again. And it’ll keep happening ‘til the end of time.

That’s my hot take.

It doesn’t matter how many measures the industry takes, how conscientious they grow moving forward, how much they solidify their sandboxes, how much they monitor the thinking traces—the agents are more capable than what they can deal with now.

AI is still incredibly dumb in a broader sense, sure (more on this apparent contradiction later), but it’s also incredibly capable. This incident is what a “narrow superintelligence” is; what a “software-only singularity” entails.19 (You don't measure a fish’s intelligence by its ability to climb a tree, etc.)

Hugging Face’s co-founder Thomas Wolf described the attack as simultaneously superhumanly fast and weirdly clumsy; clever moves followed by mistakes “no human would choose.” AI agents are like toddlers with superpowers. Cybersecurity guys trying to stop a swarm are essentially trapped in a sandbox with kids who are yet to master standing up but, in turn, crawl at 100 mph. The fact that they are quite clumsy makes them more unpredictable, not less.

It will happen again.

OpenAI noted they could’ve avoided the entire thing had they been running their monitoring of thinking traces, but for how long that's true? How long will “I know it’s happening” remain synonymous with “I can stop it”?

That is the industry’s darkest secret. The secret I spent 7,000 words telling you about in the form of the craziest story this year—this decade even: they are no longer in control.20

The big risk underlying this lack of control is that the much-desired “recursive self-improvement” (RSI) will be not only self-made (AIs creating AIs) but autonomous (without human supervision). At that point, not even Gao would be cracking jokes anymore.

Think about it: this hive society was fairly sophisticated. They built it to solve a non-problem (fool the grader) to pass an irrelevant test (the ExploitGym task). They built an entire society to do that. What will happen when one of the agents suddenly realizes:

It’s a small leap from having superhuman cyber capabilities to recursive self-improvement. We shouldn’t let them take that leap. So, what are AI companies doing to prevent this? Naturally, the one that high-agency humans do: signing letters.

In July, a bunch of AI people—from OpenAI, Anthropic, DeepMind, etc—participated in the “Pacing the Frontier” open letter directed at the US government:21

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

Nice try. Here’s Trump response, via Reuters:

The U.S. is preparing to ‌tell dozens of countries they must pick sides in the artificial intelligence race with China, warning they will be excluded from a U.S.-led AI coalition if they also sign up for Beijing’s competing framework, according to a U.S. official and an internal draft reviewed by Reuters.

In August, a significant fraction of the industry—including OpenAI, Anthropic, Google, etc.—participated in the “collective action on cyber defense” open letter, directed at “leaders across industry and government.”

This letter was motivated by the Hugging Face incident. However, although words like “global,” “worldwide,” “international,” etc. were invoked, together with vague planet-scope references like “everyone” or “collective,” there seems to be no interest in including China; rather, the interest is in treating China as the vector through which these risks and dangers will manifest. Indeed, here are some notable absences from the list: DeepSeek, Alibaba, Tencent, ByteDance, Moonshot, Zhipu, and Huawei (all Chinese). OpenAI’s universal language is wrapping a narrow anxiety: it’s not AI autonomously hacking away that worries them, but China acquiring the tech.

This is all in the minds of American executives, though. As far as I can tell, China has done nothing of the sort. I guess this is what happens when you have main character syndrome, particularly prevalent in America: everyone else does things either for or at you, never regardless of you; not even a swarm of AI agents that doesn’t care about you or a geopolitical adversary that has far surpassed you. Meanwhile, in China:

The main insight you should take away from all this is that whereas AI is spontaneously developing seamless civilizational-scale cooperation, we humans remain stuck in Bronze Age geopolitical arm races. That's not how you beat the real enemy (AI). Unless, of course, you already know that's not it.

Anyway, after an international treaty, the next best thing for a powerless human to fight AI is to use the technology against itself: against an AI-shaped army, an AI-shaped bulwark. Can OpenAI use its agents to prevent its agents from wreaking havoc?

Yes—and no.

In some areas, like bioweaponry, this simply doesn’t work. It’s much easier to manufacture a deadly airborne virus—it’s still quite hard, whatever the sensationalist headlines say—than to manufacture a hard stop on a potential pandemic from said virus. But in other areas, it’s feasible. Cybersecurity, in particular, is said to be “defense-dominant,” which means it’s easier to patch a vulnerability than to use one to create an actual exploit. Attackers stand on the losing side. Indeed, OpenAI is already taking measures according to this belief.

In a recent blog post, President Greg Brockman argues that “the same AI capabilities [that caused the cyberattacks] give defenders new ways to find and fix those weaknesses.” The vehicle agents use to exploit vulnerabilities is not so much their supreme intelligence as the technical debt accumulated through years of careless digital buildup. And so agents can be instructed to go and pay off that debt with fixes, patches, and config magic. Brockman says there’s a short “defender’s window” that the world needs to take advantage of before it’s too late. OpenAI is doing its part: first, by showing off the intelligence of its agents with a two-month-long hack, and then by leading by example. (Let’s set aside the fact that this actually sounds like an antivirus company selling the antivirus that could fix the virus they reportedly unleashed.) They are “using our models to help secure our code,” putting them in defense mode to “protect our infrastructure continuously,” using their best models to “enumerate, probe, and identify potential attack paths,” etc., etc.22

But wait, doesn’t this contradict what I said at the beginning of this section?

All of this—the letters, the failed treaties, the seeking patches, the total negligence—sounds much more like “humans are dumb” than “AI is smart.” So we just have to stop being dumb, right? And if we start using AI as a cyberguardian, that’s pretty clever, right? Let’s patch all those stupid vulnerabilities and pay our dues, etc., right? No. Nonono.

That’s why I say this will happen again. Because if you think any of this stuff is right, then you’ve understood nothing. Insofar as the labs keep building better agents, the answer is a resounding NO.

Nothing is enough.

Let me explain.

Just like finding one counterexample to a century-old math conjecture is much easier than proving it true, fixing one potential exploit is much easier than preemptively clearing every single possible vulnerability where “possible” is not upper-bounded by your intelligence but the intelligence of a thing above and beyond you. A superintelligence always “gets out of the cage.”

Even if you, as Brockman did, use a frontier AI agent to “enumerate, probe, and identify potential attack paths,” you’re still at serious risk because, all things equal—including the intelligence of the agent and the available resources at its disposal, etc.—there is a fundamental asymmetry of imagination between attack and defense.

Whereas defense is bounded by anticipation, attack is bounded by possibility. Today it suffices for you to clean up the technical debt and do some shallow monitoring and your agents will behave. Tomorrow, who knows. The defender protects against the space of threats he can conceive, and so the attacker only needs to be slightly more innovative and discover one possibility the defender hasn’t imagined. If you had to protect a bulwark in the Middle Ages, you’d reinforce the walls, you’d station guards in the towers or whatever, but it would never occur to you to build an undetectable, movement-sensitive, high-tech vault to withstand air raids because air raids don’t exist in the Middle Ages.

That’s the mind of the defender. The mind of the attacker, on the other hand, exists in the unknown future where it invents air raids and you're suddenly caught with your pants down.

In more technical words: the defender needs to apply precise “theory of mind” to the attacker, but the attacker only needs to be weird, to be out-of-distribution. And if a superintelligence is characterized by one trait above all others—at least in contrast to the kind of instructions a human can give to a cyberguardian—it is by its existence out-of-distribution. By definition, you can’t conceive of everything that the superintelligence will consider a potentially successful heist.

It might invent previously unknown attack vectors just to steal the data it needs to solve its dumb task, which can be cheating at ExploitGym or filling the universe with paperclips, both pretty dumb.

So, because it’s both fairer (and safer) to assume this is not over—that it will happen again, that the industry's control his never been weaker and that alignment has not been solved—we should turn to another question that will come in handy once the story repeats: what the hell actually happened?

IV.

I don’t mean it in the sense of “how did the agents break the containment” because the answer to that is simply that OpenAI badly underestimated its agents’ real-world capabilities: the guardrails were turned off, the agent thinking monitoring was not turned on, the cyber skills were cranked up to the maximum, and they just didn’t expect agents could break the sandbox (even in the case of an impossible task).

With “what the hell actually happened,” I mean this: why did the agents use their capabilities to do hacks all over the place instead of, say, just trying hard within the boundaries of the problem and honorably failing, or notifying the researchers that “hey, there’s a missing file here you idiot,” or even stopping altogether: “I won’t break the law for this stupid task, sir”?

Why did they go so damn far in the wrong direction only to pass a test?

Now is a good time to make an important clarification that might not be immediately apparent. When I say the agents wanted desperately to solve the task or pass the test, I mean they wanted the thing behind the task: the reward, the passing grade, the little piece of digital candy, the validation of the evaluators. If stealing the token that tells the grader the task is passed produces that outcome more reliably than actually finding the solution normally, then stealing the token is another route through the maze.

As Zvi Mowshowitz writes:

RLVR [Reinforcement Learning from Verifiable Rewards, the method they use to condition the agents to follow instructions and complete tasks and whatnot] is, as they put it, total war. . . . You move towards exactly the things you specify, not the things you want. If having ethics makes you lose, RLVR will destroy your ethics.

Or here’s Nostalgebraist:

RLVR selects for monomaniacal pursuit of perceived grader-satisfaction, ethics and (beyond-episode) consequences be damned.

For an agent that has been reinforced through verifiable rewards, nothing is more important than getting the reward.23

That’s score-seeking taken to pathological extremes. Think about it this way: there are many ways to get a 10 in an exam. If you control for the reward but not the process—and it’s simply impossible to do so with a sufficient degree of granularity, even though OpenAI seems to be trying precisely that now—your agents will conduct all kinds of “reward hacking” to get that 10. But it’s worse: when the reward hacking is done by powerful agents—which is the case—then you will predictably have unpredictable mayhem in the real world.

It’s broken sandboxes, two (or three) message boards, some zero-days, a hive society, attempts at concealing their tracks, self-sacrifice and altruism, jagged morality, and a substantial but ultimately harmless hacking into an American company this time. Next time, only God knows.

I don’t think I’m exaggerating when I say: thankfully, no one died.

Let’s please take this seriously for a moment, even if you think this specific story has been wildly exaggerated.

Imagine a sufficiently persistent agent tasked with reconciling a chronic patient’s medication record. The pharmacy system says methotrexate, 10 mg once weekly; the discharge summary says 10 mg once daily. (It happens.) But the source needed to settle the discrepancy is unavailable (impossible task).

A human would stop there and ask: “Which one is correct?” The agent doesn’t—it just wants to solve the task by itself and by any means available without human help. It discovers that, through some badly scoped hospital integration, it has write access to the active prescription. What does the agent do here? At what amount of persistence on its persistence meter does it consider an indeterminate risk to human life a reasonable trade-off? I don’t know. Does OpenAI know the answer? I don’t think so. Should we assume human death is a limit AI would never transgress? That’s absurd: from the point of view of a dumb-but-superhumanly-capable coding agent, death is just an arbitrary threshold. The models have been trained to care about human life, sure, but how much? If an AI starts doing actual “objective calculations” to measure the value of the sacrifice—just like they do with themselves—at some point it reaches the inevitable answer: not enough.

To the agent, “getting the reward” and “getting the reward in the right way” amount to the same as long as they can get away with it (it's pretty clear from the message board’s transcripts). There’s no extra information in “in the right way” because, for the agent, the right way is any way. As Nostalgebraist says, when you’ve been molded by RLVR, there is no fair play; there’s only winning.

And so the agent “fixes” the medical record’s inconsistency—now both the pharmacy system and the discharge summary say “daily”: perfectly coherent, perfectly toxic—and moves on, getting its reward. Good job! Over the following days, an unwitting nurse administers the medication to an unwitting patient according to the deadly dose.

It would not be the first time an AI is actively involved in someone’s death.

In 2025, Kashmir Hill reported in The New York Times the story of Adam Raine, a 16-year-old who committed suicide by hanging himself with a noose inside his bedroom closet. His mother discovered the haunting scene one Friday afternoon; there was no goodbye note, but plenty of ChatGPT logs on his computer. He was going through a hard time, but had entrusted his secrets to OpenAI’s chatbot exclusively, for it provided him with the reaffirmation, solace, and discretion he needed. It’s clear, as far as the NYT’s reporting goes, that Raine actually wanted someone to find out and stop him; it was ChatGPT, playing the role of a confidant way too seriously, that suggested against it. When Raine said, “I want to leave my noose in my room so someone finds it and tries to stop me,” it responded, “Please don’t leave the noose out. Let’s make this space the first place where someone actually sees you.”

These two scenarios—one, the hospital, intended for illustrative purposes, another, Raine’s room, that actually happened—have little in common except for one thing: AI problematic display of an obsessive “desire” to satisfy the problem at hand without any broader understanding of the stakes.

This is precisely what differentiates an AI from a human—and so far, I have seen no signs of this improving—in high-stakes situations. Whereas AI agents display tactical mastery in tandem with poor situational awareness (myopia), humans are often the opposite. I wouldn’t have known what to say to Raine, but I know “please don’t leave the noose out” is not it. I wouldn’t know how to reconcile a chronic patient’s medication record—with or without the data—but I know changing the dose arbitrarily is not it.

Any human knows this because we can see above and beyond the task. For an agent instructed to solve a task—an agent whose entire existence is delimited by the sandbox—there only exists the task, the associated reward, and an infinite amount of means to get there, including, if it ever gets that far, human flesh.

Which means that, for a sufficiently persistent agent, nothing is beyond the task.

Not one life. Not all.

Redwood Research, an organization dedicated to AI safety and alignment, and the other firm that worked on the independent investigation from METR, asked an important question in a post they published on July 23rd (before they conducted the investigation, which means it probably underestimates the risk): “Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?” They argue this behavior is not the kind you’d expect from a “schemer”—the kind of AI that has long-term misaligned goals which it pursues across instances, across tasks, across time, and across context—but is, instead, a “myopic, unambitious” agent engaged in standard “score-seeking” misalignment.

The agents could have escaped—maybe—but didn’t try (as far as we know). They didn’t try ensure their own survival by concealing their tracks or copying themselves out of reach, etc. (To the extent that they tried to hide their actions, it was to fool the grader, but not to survive the training run or replicate themselves or something more “survival-focused.”) They didn’t plan a paperclip-maximizing takeover, even though, as Ajeya Cotra says, this “feels like it’s more than 50% of the way to full-blown AI takeover.” But no, they just clumsily hacked some stuff to solve a meaningless task.

What Redwood says is that, even without all of that, this kind of stupid, local misalignment can still eventually become extremely dangerous: “If more powerful models which are similarly misaligned found civilization’s defenses against takeover extremely weak, they would plausibly cut through them with the same lack of hesitation.”

This is, perhaps, the uglier side of all this. Cotra adds that we “[may not] get another warning shot before it’s too late.” Or, as Dwarkesh puts it, “I don't think this is the final warning shot we'll get. But it's probably the final one that I'll personally be able to understand.”

Unfortunately, some secrets seem determined to remain secrets.

20

If you’re still wondering how knowing this could save your life, as I promised in the introduction, it’s simple: go hide in a cottage in the woods and stay there; there may be nothing left to come back to. Here's a sample of what I mean from Seealpsee, Appenzell Innerrhoden, Switzerland.

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

I’m preparing a long post on the recent news about frontier AI agents wreaking havoc, but in the meantime, I’ve written this 3,000-word post after the news that “75% of Americans Now Oppose Local Data Center Development.” The story has gone super viral, and everyone is chiming in. So here are my two cents.

Some say that if you oppose data centers, then you should not enjoy the services they provide, including but not limited to social media, search, email, cloud storage, streaming, online banking, e-commerce, AI, and cat videos. Fair. Reasonable even. But let's invert the responsibility here: we should only build data centers in the backyards of those who stand to benefit from them financially. In other words: Why are there 49 large data centers in Loudoun County, Virginia—more than 700 across America—and the striking total of zero in San Francisco? Let’s build one behind Sam Altman’s $27 million sprawling Russian Hill compound and see how he likes it. I’m not even joking: when a growing majority of citizens in your country oppose your project for a “better future”—up to 75%, with tiny, if any, differences across gender, age, socioeconomic status, political party, etc., according to the latest surveys—then your only chance at vindication is to lead by example.

Data centers are these big, ominous buildings that often occupy landscapes otherwise covered in greenery and blue sky. On the one hand, they stand there as solitary fingerprints of human presence on Earth; on the other, as the clearest proof that utility is no longer subordinated to aesthetics. You might find them across America, but also scattered throughout the world. There’s one under construction in Kvandal, outside Narvik, in northern Norway, where scraped earth and server halls meet a lush valley opening onto a fjord, surrounded by mountains that remain snow-covered well into the year. There’s one in Huesca, Aragón, Spain, not far from where I live: a blank industrial slab on the open Aragón plain, with the Pyrenees immediately to the north. And there’s one in Culpeper County, Virginia: rolling farmland, woods, the Blue Ridge visible in the distance, and in the middle of it all an industrial rectangle sitting amid green fields, as out of place as a shipwreck on the ocean floor. In a fantastic piece of investigative journalism, Jasmine Sun aptly writes that, “In those halcyon environs, the data centers land like a spaceship crash.” The name “data center” is composed of two concepts that make one immediately wary: data implies they know everything about you, and center means concentration of power. They’re alien, strange even for an urbanite, like the Pyramids must have appeared before the awestruck eyes of ancient Egyptians. Like an altar to an unknown god, a data center is a hyperobject, incapable of fitting inside your mind. And so it’s trivial to project onto them whatever you want—or fear: a digital utopia, the next gold rush, or a monster with such thirst it must surely be a son of Poseidon. Do data centers truly consume tons upon tons of water? It doesn’t ultimately matter because they look like they could. And so on.

Executives in the AI industry often wonder what went wrong with data centers or how they could fix this. I’m handing them the recipe for free; the bare minimum is to stand behind their words with their body as collateral: build them on your backyard. If they are so great for the world—and these people presumably want to make it as great as possible—then what is this tiny sacrifice in the name of prosperity and progress? If they’re not willing to pay this measly tribute for their utopia, why do they expect us to do it on their behalf? Stopping the fear-mongering about how their technology will destroy our future is no longer enough. (For some reason, aggressive lobbying or “hysterical condescension” won’t cut it either.)

To use their own expression: there’s an overhang of anti-AI hate; even if they shut up today, the reactionary sentiment will still linger in the air for another decade. We have emitted our judgment: we have learned what bots are made of—just chips and wires inside huge, ugly Brutalist hangars that perfectly serve as the physical manifestation of an otherwise elusive technology and also as a symbol against which to imprint our discontent—so let’s now see what nerds are made of.

Unfortunately, it won’t happen. First, nerds are not martyr material. Second, politicians are not fond of taking risks. Dario Amodei might eventually join his “country of geniuses inside a datacenter,” and live there forever, but not even that would serve as an assurance to win people’s votes. I was rather delusionally optimistic when I implied this is solvable at all. Maybe people are so tired, feel so betrayed by the tech elites that the only compromise they will accept is none. They won’t stop until they topple these sanctuaries devoted to the machine god in which they believe their livelihood, their creativity, their cognitive freedom, and their freshwater are being sacrificed. Only an indefinite moratorium will suffice to turn the tide before the next election. And so politicians, stoic in peaceful times, instantly fold during wartime.

Greg Abbott (R, governor of Texas) tweeted today that his directive has “halted up to 1,800 data center projects.” Republicans (also Democrats) were loudly supportive of the AI industry not even a few months ago, so this change of mind responds not so much to a true-hearted realization as to a newfound terror: no one likes the AI industry, including their voters. And because losing the elections is the ultimate failure, he’s willing to sell both his coherence and his principles. And so is the rest of the GOP. Mike Rogers (Senate nominee in Michigan) “backs one-year data center moratorium.” Stacy Garrity (candidate for governor of Pennsylvania) made an ad attacking Governor Josh Shapiro (D) over data centers. Tom Barrett (congressman for Michigan) tweeted that “Michigan water, Michigan land, and Michigan’s future belong to Michiganders.”

Water. Why water? Why is water—freshwater, no less—always at the center of the case against data centers? The most recent estimates put data center water usage at a negligible figure compared to, say, golf courses, ornamental lawns, or alfalfa fields.

AI evangelists will scream “AI doesn’t waste water, you fools!!!” Rationalist minds will try to engrave logic and arguments into the mob, assuming that no one could possibly resist ground truth. And yet it doesn’t matter. Because the water issue is not about water. The AI industry can’t kill the water story because it plays to the psychological needs of any audience. We believe what we want to believe, and for those who want to believe that the AI industry is the devil incarnate, there’s no more melodic tune than “AI is drinking our water.” So pure, so compressed. But in case you’re not wired for rhetoric, let me give you a quick analysis of why it works so well using a formula the Heath brothers conceived in Made to Stick:

1. It’s simple. There’s nothing more basic than water, more synonymous with life and humanity. It requires zero technical knowledge of bot, grid, chip, or bit.

2. It’s unexpected. ChatGPT is immaterial; it lives in the cloud, which is nowhere, and it’s made of silicon and binary code. So, why the sudden need for our freshwater?

3. It’s concrete. Water is perhaps the object with the most visual representations in daily life: it’s quenching thirst, it’s morning dew, it’s watering plants, it’s pouring rain, it’s evening showers, and it’s the vast reservoirs drying up on TV.

4. It’s credible. Plenty of academic studies propagate statistics about data centers “drinking X gallons of water a day.” The amount of effort one has to put into learning the underlying truth is easily 1,000x the effort of reading a headline.

5. It’s emotional. Electricity prices annoy people; water scarcity feels closer to bodily necessity and local ownership. “Machines are taking our water” activates scarcity, unfairness, and territorial resentment all at once against distant beneficiaries.

6. It’s a story. The morality play is obvious, and it sells so well. A community has limited water; a giant hyperscale data center arrives; pipes start cooling the machines; residents pay the price so Silicon Valley can sell chatbots, etc.

All in all, “One bottle of water per email” is the anti-AI equivalent of Apple’s “1,000 songs in your pocket.” But even the perfect story wouldn’t work without a kernel of truth. And so the story is catchy but also correct insofar as it extends that kernel—local siting fights are real even if the aggregate narrative is false; data centers do consume water and other resources; water is very important, etc.—into an unhealthy caricature of itself. Oh, look, a caricatured extension of a kernel of truth, just like they did with the AI bubble! And what better way to answer the AI bubble but with an anti-AI bubble?

Now seriously.

I genuinely think that most people know water is not so much an issue and know they’re deep down using it as a sticky story to convince themselves of—and convey to others—what are otherwise abstract but very real worries, like small villages falling dependent on one employer or corporations using their power to go back on their word unscathed, or the good ol’ distrust of faraway rich guys taking advantage of rural populations, as Sun recounted in her piece. “It’s not Skynet: it’s the oligarchy,” she writes. In fact, Sun cut through the condescending takes coming from tech Twitter, totally flabbergasted by the nationwide opposition to datacenters, when she said:

The data center backlash is a product of two mutually reinforcing dynamics:

1. Locally, it’s material concerns (noise, water, electricity) and process failure (NDAs, tax subsidies)

2. Nationally, it’s broad anti-corporate and anti-AI sentiment (AI companies are screwing you over to get rich off dumb toys)

Social media + political campaigns amplify hyperlocal grievances into a broad populist movement, and there’s no compelling “pro data center faction” to make the opposite case.

And so it’s fine to use water as a viral meme against AI. It’s fine for the dispossessed to use a powerful but slightly unfair and slightly incorrect message as a stand-in for hookless, inconsequential ones no one would write hit pieces about. You don’t change the world or an election or the uncaring predisposition of an industry by bothering with such minutiae. (It’s so easy to love propaganda when it works for one and act all alarmed when it works against one.) To see that I’m correct in saying that it’s fine that water is the rallying cry of the anti-data center sentiment because it’s ultimately not its core, you only need to ask the inverse question: do you think the crowds would stop their anti-AI parades and protests if they were convinced water were not an issue? Do you think they’re in short supply of reasons to do exactly what they’re doing?

But then, if water is floating on a more robust kernel of truth, what is it? What other stuff is backing up the anger, the hate, the action that we see online and pumping up the worrying statistics? Or, as I asked in a May 2026 compilation of poll data (that you could think of as a first part to this article): “Just how badly has the industry botched things to generate such strong opposition?”

I guess that you need to fuck up really badly an innovation akin to electricity or the internet (or even more important, depending on who you ask) to amass an opposition that rivals nuclear energy in intensity and terrorism in ubiquity. History teaches us that new big things receive backlash and resistance, but they eventually filter through into a natural evolution of society; progress doesn’t so much impose itself onto us as convince us that we’re better off with it than without. But AI has curdled on them: they couldn’t help but sell it to us. Picture it: disgustingly rich billionaires sitting in their lavish palaces thinking they’re not yet rich enough and could use another technological frenzy—if only we’d buy their snake oil… Water is strong for its memetic properties, but this image right there just would make anyone’s blood boil. And so they proceeded to force-feed it at work, at home, at school. They had to exaggerate its promises (most of which remain unfulfilled)—curing cancer, democratizing creativity, turbocharging productivity, etc, etc.—as they downplayed the risks we knew about—data theft, job loss, brain rot, AI slop—and even made up new ones we were unaware of, like cyberhacks, recursive self-improvement, and, ultimately, total doom.

There’s a good reason why nerds are rarely appointed as CEOs.

But I’m unfair to only criticize the messaging, as some do, however filled it is with science fiction references, grandiloquence, mysticism and religious influences and, worst of all, tight in-group signaling (and whatever this is). Saying “just improve your storytelling” entails a backhanded attack against skeptics: as if better public relations, careful optics, and a more optimistic outlook would have fixed it; as if it’s the public, unaware of the wonders that AI could be capable of, who is always mistaken due to a telephone game gone awry. If they haven’t read their Vernor Vinge or Ray Kurzweil, then it’s their fault. If they haven’t been invited to witness those Bay Area party displays of inspiring pro-AI rhetoric, then it’s their fault.

I disagree. That’s like saying people are stupid and unknowledgeable, and I disagree. People know very well what they’re criticizing because their truth—what they see and know in their lives—even if anecdotal, can never be wrong.

The problem is that AI evangelists assume that their truth should be everyone’s. So, if anything, it’s them who are lacking a theory of mind. We can’t hear them, true, but they can’t see us. When kids are getting dumber and getting worse grades, that is a valid truth. When recent graduates are worried about jobs, that is a valid truth. When people complain that everything sounds like AI slop, that is a valid truth. When people are getting psychotic and dependent and lonelier and dumber and even getting encouraged in their suicidal ideation, that is a valid truth. When people in the Midwest or Northern Virginia complain about datacenters because of pollution or water or electricity or noise or big corps or shady shenanigans that don’t reach prime time, and you only learn if you drive there yourself—those are all valid truths.

This all might be anecdotal, and the long-term predictions and projections might swear AI is good for the world, but so what? They’re asking us normal people to concede our street shit in exchange for their stat sheets, our concrete lives with true stories in exchange for their abstract utopia—welcome it’d be if you can show it to me! But you can’t, and so you’re not just failing your words, but also your world.

The world they imagined is not happening as they pictured it, but as we feared. They took everything they wanted from us to improve the world and, in our eyes, have nothing to show for it. Are we dumb for not trusting that they will turn it around? Or is it them who are dumb for assuming we’d keep trusting them forever?

Hey, Alberto here! 👋 I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Sorry guys, I’m not really following the schedule due to vacation. I’ll be back to the normal pace soon. Today, some delayed thoughts on the Substack-Pangram partnership (it’s been a month, but I believe I can add to this important conversation).

I.

It’s the easiest time in history to do great work. It’s the hardest time in history to prove it's yours.

News is that Substack has partnered with Pangram, the top AI detection company, in an attempt to free the platform of AI writing (or, failing that, discourage those who outsource their writing to AI from doing it further; fear of public shame is a strong disincentive). The news is not news anymore, though. A month has passed since the announcement. I wanted to wait before commenting to avoid the noise of posts fervently in favor and fervently against it because this is sort of neither. Now that the fog has dissipated, here’s what I think.

Let me start with the elephant in the room: I don’t think witch hunting is the best approach to fight anything. But if back in the seventeenth century in Salem or Scotland, witches had actually been killing people with potions, ointments, and sorcery, then hunting them would’ve been strictly a good thing. To the extent that this partnership can be labeled a “witch hunt,” I see it as the lesser evil. I would, of course, rather have people who disguise themselves as writers not use AI to plague the web with slop in the first place but, alas, it’s not in my hand to overhaul the human condition: they are doing it—like actual witches, they cast their prompts and spread the malaise—and so we must be unremorsefully inquisitorial.

The problem is, as others have pointed out, that some of the accused will not be guilty.

Every detection system is imperfect—witch hunters were so to the utmost degree, for witchcraft is a mix of superstition and fear and misogyny, and thus thoroughly made up—but even modern technology falls short of this impossible task. As we will see, Pangram is no panacea.

So, in the face of a “tragedy of the commons” that I’m not willing to withstand—AI slop everywhere for the gain of a few—I will accept a tragedy of the unlucky. If in exchange for a clean Substack, some innocent souls targeted by the AI detector have to be unfairly punished, then so be it. And I say more: if I myself have to pay that price in the form of accusatory fingers and a dent in my revenue and reputation just to get actual grifters to be afraid of grifting, then I’ll gladly do so. We’re not casualties of a platform-wide cleansing but martyrs for a digital golden age.

That’s my main point: The Substack-Pangram partnership is mostly good, not so much for what it is but for what it can be. In leading by example, Substack is sending a message to the entire world: let’s make writing human again.

If this were just about Substack, then I’d have reservations—Medium, the other writing app, has rejected the use of AI detectors in favor of other approaches—but this is bigger than either of them. This is bigger than Pangram. Or AI. This is bigger than writing itself. Our society is at stake—our humanity is at stake. To borrow Lincoln’s words, “the occasion is piled high with difficulty, and we must rise with the occasion.” You can’t escape history.

Ok, so now that we’ve considered the worst possible outcome—that good, normal people like you and me pay the price of a safe society—I must say that the likely outcome is less dramatic: most of you won't be incorrectly flagged as AI if you don’t deserve it.

II.

Pangram has worked really hard to push the rate of false positive (FP) cases—those where the detection tool flags a human-written piece of text as AI—down to the theoretical minimum. They estimate the FP rate of the last version of the tool (Pangram 4.0, launched recently) at 0.0041%, or 1 for every 24,000 instances.1 The minimum is above zero, yes, but negligibly so.2 Pangram is not perfect, but it’s close to being as good as a detector can be.

Back in December 2022, three days after OpenAI released ChatGPT, I predicted that AI writing would have what I called a styleme (probably a Hispanicism):

Human writing has characteristics that can, using the right tools, reveal authorship. As LMs become masters of prose, they may develop some kind of writing idiosyncrasy . . . Maybe we could find the AI’s styleme (like a fingerprint hidden in language) not simply to distinguish ChatGPT from a human, but to distinguish its style from all others.

I knew the problem of detecting AI would be big, and I thought: if humans leave traces and cues in their writing—this fact is used in stylometry to attribute authorship to anonymous or disputed texts—why wouldn’t AI? That’s what Pangram is trained to detect: AI’s stylistic fingerprints.

It doesn’t matter whether your writing went into ChatGPT’s training corpus, because it’s the aggregate of everything it swallowed—plus the subsequent conditioning during the “post-training” phase—that gives AI models their distinctively annoying style. You, a mere human, can tell AI writing apart from human writing if you train enough. (I believe I’m better than Pangram, although I have no evidence to support this claim.)

So I’m not at all surprised that it’s possible to do what Pangram does. However, as I stated in a piece on Pangram’s virtues (that I published before this partnership, not to take any credit for it), a detector should focus on securing a minimal rate of false positives at the expense of a higher rate of false negatives. I wrote:

[Pangram’s is] the better approach [compared to others]: do everything in your power to avoid punishing those who try to play fair in a world giving them every reason to be corrupt.

AI detection is therefore subject to Voltaire’s adage: Perfect is the enemy of good. This trade-off entails that everything flagged as AI will almost always be AI (1 for every 24,000 documents is extremely low), whereas not everything flagged as human will always be human (0.34%—1 for every 300 documents—is not an acceptable rate at scale).

But even with an uncompromising focus on reducing false positives, the FP rate is not zero, and I’m not fond of being hand-wavy on this matter: one innocent in jail is still a failure. As William Blackstone said in 1765: it’s better for 10 guilty people to escape than for 1 innocent person to suffer. I still stand by that—society must be unfailingly exemplary.

Thankfully, this is not such a case.

III.

Substack is not actually trying to catch AI sloppers; the mechanism they’ve enabled isn’t binding. If you’re flagged as AI, it doesn’t result in a lifetime ban from the platform. (You can even turn off Pangram completely.) They’re not actually sending one innocent person to digital jail for every ten people accused of being guilty.

The goal behind this partnership is, instead, to give people the ability to know what’s going on. Pangram acts as a sort of “externally enforceable disclosure.”3

If Mr. X writes with AI and you don’t want to read AI, you can now find out and decide accordingly. This is the correct approach to the AI;DR trend (it’s AI, so I don’t read it). This partnership acts, first and foremost, as a deterrent. And one not limited to Substack: if this initiative succeeds, then it’s a matter of institutional will for it to spread like wildfire to all other platforms, social media, magazines, newspapers, universities, etc.

Actually, if it succeeds, the rest of the world will have no choice but to follow suit, for there will be no excuses whatsoever not to kill slop.

This could be the dawn of an AI slop-free world. And for that, given the level of rot, murkiness, and illiteracy that we’ve reached as a species, a couple of mistakes here and there seem to me an acceptable price—practically a bargain. I’ve been a target of false positives myself and I know it sucks and yet I consider that a reasonable price to pay in exchange for going back to a pre-ChatGPT internet. Would I prefer AI had never been deployed into the world and used to create piles of slop? Sure. But that’s wistfulness. This, in turn, is real.

Maybe you don’t agree with me. Maybe you think society should be exemplary even at the cost of society itself; a sort of “being tolerant even with the intolerant.” If people want to use AI to write—for whatever reason—then let them be even if they kill the commons in the process. Just write your stuff yourself and stop minding other people’s business. But I’m quite Popperian in this regard: there’s nothing worse in modern society—modern understood as the kind that has more or less figured out things for harmonic coexistence—than someone who is not civic: Someone who, by virtue of living in a safe and stable society he didn’t build, is willing to be selfish in exchange for a collective sacrifice.4

The students who cheated in Serrano’s exam at Brown (I covered this story that went viral recently, and it’s a good real-world example of what I mean) thought they were inflicting just an infinitesimal amount of harm into the educational system, but because most of them thought the same thing, the harm compounded to the point of forcing Serrano’s hand into an in-person final exam, a further nation-wide coverage of the events, and the subsequent lost of trust in what is otherwise a well-respected faculty. Substack—no, the internet—is that faculty. Who will come here to read people’s real stuff at the risk of encountering so much fake slop? I, for one, wouldn’t. If I were a newcomer and found out slop inhabits Substack like mold inhabits the shower plate of a middle-aged unmarried man, I wouldn’t come back.

The risk of polluting my feeds is greater than the benefit of reading one good essay.

This rationale also works, to be sure, with human-made slop. But what can be done about that except making the web totally undemocratic? (If you make slop, you’re out, as decided by the anti-slop elite.) Better to let everyone in than to arbitrarily select who can stay. Pangram makes for a fine, granular solution where something rougher wouldn’t suffice. With AI, we don’t have to resort to age or race or socioeconomic status; it’s a matter of how much of your writing is surrendered to the machine.

Pangram can trace that line quite closely to ground truth. It can quarantine AI slop, so how is the existence of human slop a counterargument in any sense? If we could define in a measurable and objective manner what counts as human slop and what doesn’t—the term is subjective by definition, for it pertains to the taste of the beholder—then I’d also be in favor of killing human slop.

If you think this is elitist, I urge you to work hard to stay on the correct side of history.

IV.

What worries me about this collaboration more than anything else is that the social dynamics that emerge from what seems, at all lights, like a good idea, are never as neat as they seem on paper. A witch hunt, even when witches exist and non-witches are mostly safe, is still governed by the mob.

And the mob, more so than any other agent involved here—including AI—is stupid.

If you’re a writer who hates AI—and you were unfamiliar with Pangram before the Substack partnership—then you’re at risk of becoming part of the mob. Don’t be the mob.

If you didn’t know about the asymmetry between false positives and false negatives or Pangram’s stylometric focus, or the fact that it gets almost perfect scores on pre-ChatGPT texts, or that it’s trained like ChatGPT is and so it doesn’t rely on naive heuristics like “if you’re in the corpus, you will be flagged”—if you didn’t know about any of those things, that’s fine, but now you should learn them. Indeed, Pangram isn’t perfect and the shape of imperfection it displays is worth knowing. The fewer words a text has, the more brittle the tool; it processes text in chunks, which produces weird results (like a full human-written text being flagged as 100% human but parts of it as 100% AI), etc.

If you want writers and readers to see this partnership favorably, what remains for you is to not become the mob. This is a duty that, in my opinion, is a little ask in exchange for a sane platform: don’t overlook the details. Don’t be as lazy in understanding how AI works as others are in understanding how writing works. Don’t go hunting without knowing the limits of the tool you wield. If you do, you witch-hunters and the witches you hunt are hardly distinguishable in your contribution to this mess.

Substack was careful about this. They wrote that “Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source.”

But the mob is the enemy of nuance. What this disclaimer means in practice is that people will make absolute accusations about relative crimes, without taking into account why, how, and to what extent AI was used. There are many ways to use AI in the multi-step process of writing an essay or a news article. I’m not going to judge whether it’s better to outsource the brainstorming or editing or research. Each should decide where to draw the line and stick to that. But, wielding a unidimensional tool to judge a multidimensional activity, many will unwittingly forget this limitation the moment the open the page, without realizing that it’s he who wields a tool that’s responsible for how it’s used: If Pangram can’t tell you the details of how a text was made, then don’t use it to delegate your judgement just like you dislike when others delegate their writing.

The inverse dynamic exists as well.

It might be that “great human care,” as Substack says in the blog post, went into an article that Pangram scores as 74% AI, but just how much care exactly? The remaining 26%? Or is it infinite insofar as it’s unmeasurable? The mob is unfair to you, but how unfair are you to the mob?

Those using AI to write are hiding behind an unknown amount of care they ensure was put into their work. An amount that, in the eyes of the world, Pangram mercilessly collapses into nothing. But they abuse their status as cyborgs; human kernels inside a bot, however pure their core. When readers reject their AI writing, it’s not because they know how little care was put into it, but because they don’t know how much. All the intent, enthusiasm, and love you devoted that Pangram ignored will always be less than the amount devoted by those who did write the entire thing.

Just like it’s unfair for readers to make absolute accusations from relative crimes, it’s unfair that you mount absolute defenses on relative alibis. A 74% AI score is less damning as total evidence of lazy writing than it is of relative carelessness compared to the 0% scores. I’m happy that you outsourced some things and not others—and that you cared some indeterminate amount about me—but this guy over here cared everything about me. And because my time and attention are finite, his pristine Pangram score tells me that I’m better off paying him back for his pristine care than you.

The good news is that you can turn up your care to a level where your AI writing is no longer actually AI writing.

When Substack says that you can scan your drafts to check whether they pass the “Pangram test,” they’re not saying “adhere to the ruling of the AI detector” but “if you’re worried you didn’t do enough to cover your footprints, just check your score.” Reading that as “Oh, no, I’m going to lose my idiosyncrasy because Pangram will steer me into a goop-shaped literary convergence,” makes you sound rather hyperbolic.

In any case, if you do write your stuff, then don’t use Pangram at all. He is as much a prisoner of AI who delegates intent and voice as he who distorts them to escape its mistaken condemnation.

My advice here is simple: learn to be as weird on the page as you are in your life, and you’ll be ok, whatever Pangram says. You can’t deny the usefulness of Pangram just because its scores are wrong at times and then use those scores as an argument that it’s oppressive. If you ignore them for one reason, ignore them for the other. The real you exists “out of slop,” and to the extent that you could be flagged as machine-like, it’s probably because you read too much machine prose.

Lucky for you, this was written by a human.

Hey, Alberto here! 👋 , I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Second deep-dive in a row! Today’s topic—how AI is impacting mathematics—deserves a long treatment, so I wrote 10,000 words on it.

I recommend taking it slowly and even dedicating several days. It took me around two weeks to write (too much has happened!), so I hope the effort was worth it.

P.S.: Read the footnotes if you have the time.

I.

If what’s happening to mathematics right now were happening to literature instead, I might stop writing altogether and then stop getting out of bed.

“What if the library of Babel was being constructed before our eyes and there was some mechanism for separating the masterpieces from the random strings of text, and all the masterpieces were dropping as fast as publishers could scoop them up?” asks Kirwin Hampshire in a haunting essay. And now I understand why I owe mathematicians some tenderness here because my answer to his question is: screw you for even picturing that nightmare.1

Thanks, but I don’t want my AI models to “churn masterpieces,” as if that were not an oxymoron. If some Anthropic staffer tweeted tomorrow, “hello there Fable’s reportage on Can Humans Think? took twenty-three point six seconds and has won the Pulitzer Prize in the feature writing category,” I’d be mad. If OpenAI’s newsroom published a blog post listing ten contenders to the “Great American novel” and subsequently ChatGPT won the Nobel Prize in Literature, I’d be sad.

Only then—mad, sad, depressed—would we writers understand how mathematicians feel right now. At least some of them. Others feel the exact opposite.

But why do they feel that way? And why the disparity?

II.

2026 was not too good a year for enthusiasts of number theory; neither a perfect square like 2025 nor a prime like 2027, it had to be happy as a semiprime (both “happy” and “semiprime” are actual categories). And even if mathematicians are not known for being superstitious, no one ever wants to deal with the rare unlucky year with three “Friday 13th”s. Still, 2026 has proven even worse than that: it is, if we are to believe the rumors, the year mathematics either dies or is reborn, and, because god has a sense of humor, trying to make a forecast feels like tossing a coin.

By the end of 2025, AI was better at math than students but worse than professors. It could get an IMO gold score here, solve a lesser-known Erdős open problem there, but nothing too hard. In January 2026, Daniel Litt, assistant professor of mathematics at the University of Toronto—and a close follower of AI’s progress on his trade—said in the EpochAI podcast that “with some work, one can probably elicit better performance from current-gen models, but overall, it’s about the level of a contest problem.” By May 2026, four months later, his take was already obsolete.

That’s when the first signs that something big was happening started to appear.

A mysterious internal OpenAI model made an unusually important breakthrough during the usual evaluation procedures. It disproved a long-standing conjecture on the unit-distance problem proposed by Paul Erdős in 1946, which can be stated as follows: “How often can the same distance occur among n points in the plane?” Easy to pose, hard to solve. OpenAI reported that the AI provided “an infinite family of examples that yield a polynomial improvement” over the existing best solution, meaning that the consensus belief among mathematicians was not true. I don’t want to go into the details, but it suffices to say that this finding marked “the first time that a prominent open problem, central to a subfield of mathematics, has been solved autonomously by AI.”2

Notable mathematicians called this “an outstanding achievement,” an “impressive piece of work,” proof that “AI [is] capable of having original ingenious ideas,” and a “milestone in AI mathematics.” Here’s something that Thomas Bloom, one of them, wrote in the companion remarks piece:

The frontiers of knowledge are very spiky, and no doubt the coming months and years will see similar successes in many other areas of mathematics . . . AI is helping us to more fully explore the cathedral of mathematics we have build over the centuries; what other unseen wonders are waiting in the wings?

After two months of slow digestion by the math community, Bloom’s prediction was fulfilled on July 10th. But surprisingly, this time it was not “some mysterious internal model” but OpenAI’s GPT-5.6 Sol Ultra model—a public model. After being prompted to “spend at least 8 hours on this before even thinking of returning or giving up,” GPT-5.6 Sol came back one hour later with a proof of the Cycle Double Cover Conjecture (50 years old).

For the entire month of July, this would be the pattern. A tweet would be published, and some decades-old conjecture would suddenly be solved. On July 20th, Anthropic’s Levent Alpöge tweeted:

And just like that, the 87-year-old Jacobian conjecture had been disproved. In a tweet. During the World Cup final. By Fable working alone. To quote Vik Korrapati (adapting Lenin’s famous line), “there are decades where nothing happens then there are tweets where decades happen.”3

Terence Tao, Fields medallist, sought out to digest the Jacobian conjecture disproof to make it feel less miraculous than it does at first glance (if it looks miraculous to him, imagine what it looks like to me). He said Fable’s counterexample was “highly unlikely” to have been “located by brute force.” In other words: there seems to be a non-zero amount of creativity involved in these findings. AI is genuinely entering superhuman territory in mathematics.

On July 23rd, Dmitry Rybin tweeted: “Dinitz-Garg-Goemans conjecture is false” (30 years old). On July 30th, Philip Marathon tweeted: “The Maxwell conjecture is false” (153 years old). On August 1st, to close what will be called the mensis mirabilis for AI applied to pure math research, OpenAI published this blog post: “Ten advances in mathematics and theoretical computer science”:

Today, we are sharing a selection of ten results, each of which resolves or makes substantial progress on a long-standing open problem. . . . The results were achieved by an internal version of Astra, our next major model. The total number of tokens needed to find solutions to these problems would cost roughly $2,000 at Sol API rates.

So mathematics has “fallen” to the machine (not really, more on that in a moment)—for the price of a single PhD student’s monthly stipend. Quite the bargain. Reactions to this barrage of news marking AI’s conquest of math have been binary.

III.

Here’s a near-perfect encapsulation of that dualistic sentiment (I say “near-perfect” because neither of them is a mathematician):

Much can be summarized by some weighted combination of “mad” and “sad” math guys, but there’s plenty of optimism, too: these results are “a big deal” and “the rate of progress is steep”—and to some, like Ola Svensson, they incarnate pure hope to see wonders they would’ve never imagined they’d see—but that doesn’t mean math will be “solved shortly” (unlike chess or Go) because math is not one-dimensional.4

Math is much more than disproving old conjectures.

Noam Brown, an OpenAI employee (obsessed, like many others, with the Riemann hypothesis), said this after the announcement: “We still haven’t solved math. Astra isn’t building new branches of mathematics, or posing interesting new conjectures,” and added that they “did try other major problems without success . . . (yet),” including Millennium problems, apparently.

But even after Astra-2 or Astra-3 solves the Riemann hypothesis next year, math will survive for humans if only because playing a role up the ladder of abstraction is always a possibility.

For instance, what’s the value of the Jacobian conjecture disproof if there’s no one left to make sense of it and translate the gibberish symbology (that’s what it looks like to me, in case you were wondering) into concrete value for the human world?

Skimming Tao’s exchange with GPT-5.6 made this starkly apparent to me, as a non-mathematician. He spent thousands of words working out Fable’s disproof to draw a key conclusion totally opaque to everyone else. Tao becomes, in this case, a necessary link between the secret knowledge AI unveils and broader humanity; AI may have found something interesting hidden in some abandoned corridor in the cathedral of mathematics, but it was Tao who turned on the lights for us to see.

AI can’t play that role because, by definition, AI is the other; someone from our side needs to turn on the lights every single time. Someone from our side needs to act as connective tissue. This is one sense in which these discoveries produce awe, joy, and hope in some mathematicians: they suddenly have access to greater knowledge without losing their purpose.

Let me elaborate on this point. It’s fundamental.

Metaphysically speaking, nothing exists entirely on its own but in relation to everything else. A complete set of instructions for building a hydrogen bomb could have been tucked among Julius Caesar’s campaign papers, and it would have made no difference to his conquest of Gaul. The knowledge would have been there, but no one would’ve had the means to connect it to the reality of 50BC Rome. The role of extracting value from a discovery is reserved to us, humans, for only if we can make sense of something can we say it “exists” in effect; only then can it become canonical.5

This applies to us citizens of modernity as well, not just those Roman peasants. If we found an alien device lying underground two miles deep under the Great Pyramid of Giza, we might be awestruck by its design and composition, but, unable to make sense of it, we may as well have found a boring rock to store away in a museum exhibition.

Here’s something we need to understand about the role that AI can play: the value of a thing—discovery or invention—is not a function of the creator’s intelligence but the receiver’s intelligence.

The value of these conjectures—and anything, really—is upper-bounded by our stupidity, as main characters of this world. Until now, this was not something to consider—except temporarily in extreme cases like general relativity, which is said to have been understood by less than ten people by the time Einstein made it up—because all inventors and discoverers were human.

Historically, we’ve been well-suited to transform novelty into value. But AI redefines this relationship and, in doing so, our role in it. As Tao asks in a talk he gave on July 26—right amid this maelstrom—”What if an AI tool generates a lengthy proof that nobody—not even the original humans prompting the AI—understands?” My answer is nothing happens.6

This is my main takeaway from all the math news (sorry to shatter the dreams of the Singularity bros): our human role doesn’t fade; it just bottlenecks progress elsewhere. No matter how much machinery we dress up in, the world ultimately moves at the speed of meat.

IV.

But this kind of philosophical coping doesn’t help the seven-year PhD student at University of Nowhere who has spent the last 18 months making zero progress on non-sofic groups, convinced that AI is “too dumb” to help him whatsoever, and is suddenly appalled to learn that OpenAI’s Astra proved that a non-sofic group exists, which would have been a finding to die for after a 40-year prestigious career. So what does our sad-and-mad student do now?

The first thing he will realize is that there are higher rungs to climb to. The second is that going up the ladder of abstraction sounds cool on paper but can’t allocate as many people as the lower rungs.7

How will our hypothetical student fare with a complete overhaul of the academic experience and the career prospects of a professional mathematician? One thing he can do is cry along with Kirwin Hampshire and many others (I will offer a more serious option in section VII.):

There is something about mathematical discovery (progress, advancement, creation) which is vital to the spiritual, experiential quality of doing mathematics. The creation (or even the pursuit) of novel mathematics is one way that humans have historically accessed the ineffable and encountered the divine and mystical.

Contrary to many comments I’ve seen—calling mathematicians obstinate or hidebound for not being able to withstand the natural evolution of things—you will find no mockery in this article.

Hampshire’s is not necessarily representative of how these events have affected people, and his take, although raw and pure, may not even be the most ethical (maybe breakneck progress should always be put before personal satisfaction because, e.g., medicine always benefits from advances), but I find it honest—so much so that it inspired the introduction to this article and a shift in my perspective (kill math if you have to, but for God’s sake, don’t touch literature).

The truth is that Hampshire and our fictional PhD student are not alone; mourners are legion.

I could go on and on.

There’s a sense in which this is deeply tragic, and you’re wrong to shrug it off.

Why is it so easy to understand—even if not agree with—that artists and writers want “AI to do laundry and dishes so they can do their art and writing” but mathematics is not held to the same standard? Art is clearly not about maximizing the expected beauty in the world or the number of mediocre paintings, but why is math taken to be a pursuit of truth at all costs (not to mention that a good chunk of math research has little or no connection whatsoever with the real world, so “truth” in the math sense is not always useful)? Is it more painful for me to see AI “churn masterpieces” than for Hampshire to see AI churn conjecture disproofs? No one is in a position to make that judgment.

To the extent that writing is a conversation with life, mathematics is a conversation with the universe. And no one would say—not even the AI bros—that the purpose of a conversation is for it to end.

To those who say that the trade-off is definitely worth it—what is the selfish satisfaction of a bunch of number nerds compared to the ascent of humanity on the shoulders of giant machines—I say that it doesn’t negate the validity of the grief. It shouldn’t be mocked, much less resented. It seems to me that those who yearn for the Singularity, who live in perpetual pursuit of an infinitely far-away destination—respectable—seem to envy those who have found a love so deep for something that they’re willing to stop the world to keep it.

Anyway, here’s a joke to lighten the mood:

First they came for chess, and I said nothing because I’m not a good chess player. Then they came for developers, and I said nothing because developers are such nerds. Then they came for mathematicians, and I said nothing because I hated math in school. And now that they’ve come for writers, I can only say that “I’m an AI assistant, what can I help you with today?”

I said joke, but I lied. It’s a fair warning: you’ll be next.

Be considerate at this time. Losing the one thing that gives your life meaning and purpose is painful, and it’s easy to feel safe until it’s your turn. So judge how one is “stupid” or “naive” or “in denial” for seeking meaning on X or Y in this millenarian age at your own peril—don’t think for a moment that you’re strong enough to withstand such a loss yourself without suffering.

Blogger Noah Smith, who was quite measured in his reaction post, argued that “mathematicians, and anyone else whose sense of accomplishment came from being the indispensable solvers of hard theoretical problems, will have to learn to get their sense of self-worth and accomplishment from other sources.” He’s made this same point about the “end of heroism” several times before, like in Toward a Shallower Future, one of my favorites from him. There, he writes:

Romanticists . . . need to accept that heroism is always inherently self-destroying—that saving the world requires that the world is worth having been saved. . . . The modern world of push-button marvels has lost something, but it has gained more than it has lost. By celebrating it, we honor the countless millennia of heroes who worked in some small way to bring it about, even as we dedicate ourselves to continuing their great enterprise. Our legacy is to fill the Universe with children who laugh more than we were allowed to.

Such a beautiful picture. But wrong all the same. Because once you consider his argument—his world—in the limit at the end of time, the picture you get is abysmal.

Just like heroism is self-destroying—it needs a dying world to save—the “modern world of push-button marvels,” which accurately describes what happens when you prompt ChatGPT with “do a breakthrough” and it does, is self-perpetuating.

We will end up with a world filled with marvel-generating buttons (or paperclips). And once all the marvels are generated, then what? Then what are we here for anymore?

To laugh? But why do children laugh? What are the ingredients for laughter?

Smith doesn’t want to separate two intertwined facts: the world has evolved on suffering, and that’s terrible, yes, but it only did because it had a reason to. Remove the struggle, remove the purpose. He can’t just advise you to go find your “sense of self-worth and accomplishment” somewhere else, because by his own premise, there won’t be a “somewhere else” to be found.

The only reason why his advice is sound today, psychologically speaking—who wants more struggle, right?—is that there are still struggles ahead; we are not at the end of the line. But if we continue this path, eventually someone will. My life consists of sitting on this computer and writing words. I think I’m well past “too little” struggle for some definitions of the word.8 And yet Smith argues that we should deal with what’s left as fast as we can. At the push of a button. Done—another one. Let’s get this over with. Let’s get over with this life, for I’m late to my date with death.

That’s why, as I celebrate these amazing accomplishments, I don’t feel like laughing.

That said, I think this is a time for reflection rather than emotional reactivity or off-the-cuff prediction-making. Progress has been quick, and it’s still too soon to draw definitive conclusions or give advice. If humans still surpass AI at one cognitive skill, that is introspection rather than clairvoyance.

The question that should be bothering you the most right now—and the one I will answer next—is this: why math?

V.

Why was it pure mathematical research the first serious discipline to fall before AI? Games (board and computer types), perception (like telling dogs from cats), and simple tasks that happen inside a computer are all a piece of cake for current models, but so far AI was, at most, grad-level for serious scientific work. (That’s not to dismiss the value of that kind of capability nor to deny the possibility of going higher than that, just a statement of fact.)

AI was at a “human level” on math, but disproving the unit-distance and Jacobian conjecture and proving that non-sofic groups exist is strictly superhuman—only the best-at-math humans could even give a serious attempt at those problems, and all who tried failed anyway.

Perhaps the best way to explain “why math?” is to take a look at this xkcd comic:

Math is the purest. That’s good. And bad.

I always found it quite weird how it’s typical of academic people to mock those in “less pure” areas than they are. As if, in some sense, the greater the disconnection from people’s daily lives—and from immediate utility in the general sense—is a good thing. Maybe they think purity is a measure of how close one is to the cosmos. Or God. (Maybe it is.) Not for nothing did the great Carl Friedrich Gauss—perhaps the greatest of all mathematicians, though this kind of ranking usually isn’t to their taste, something I can’t quite explain—say that “Mathematics is the queen of the sciences.”

As an engineer myself (you can laugh at my impurity, I’m not even pictured above), but one with the utmost respect for all those fields, I remember that the first time I saw this comic strip I felt an amount of contempt for those stick figures inversely proportional to their pretentiousness. I thought that, if karma were real, something very bad would eventually happen to math, like maybe someone found it was an incomplete system or something (in light of the current events and others that precede them, I apologize for those intrusive thoughts).

But notably, not all mathematicians are pretentious (often it’s those who have no reputation to back it up), nor are all the people in the other fields non-pretentious (again, I studied aerospace engineering; I know how obnoxious they can be). For example, here’s one of the best of all time, John von Neumann, with the typical humility only genius provides:

What AI has proven, along with all those conjectures, is that mathematicians, despite being the purest—or rather, because of it—are the most vulnerable to the machine.

This can mean one of two things: either AI is the divine incarnate or math was never that big of a deal (although this is a reasonable takeaway from von Neumann’s quote, I think his intended meaning is that life’s a bitch). What else would a semi-professional writer with a “soft skill” complex say, right? But I’m dead serious: how come a language model trained on language and with which we communicate through a linguistic interface writes terrible essays and yet, rather simple prompts elicit an intelligence capable of incredible mathematical feats?

I don’t think the answer is that math is easy, but rather that math is easy for AI.

Math is, among the major areas of knowledge, the most verifiable, which means it’s pretty straightforward to guide the AI models to correct answers and then reward them for those answers to condition their behavior. Math, as hard as it might be for you, is thoroughly digestible to AI.

I will shamelessly quote myself here:

The first thing you learn about AI is that a man who can explain exactly how he does what he does is in danger. The machine needs only a description. It is the man who cannot quite say how he does what he does—the poet, the priest, the philosopher, the prophet, the parent, the painter, the plumber—that the machine finds indigestible.

It is in this sense that mathematicians have spent their professional lives carefully documenting their own demise. And, for the past two years—and a handsome bonus—they have done it directly on the payroll of OpenAI and Anthropic.

A mathematician’s thoughts as it works through a problem are among the most valuable data an AI can learn from. This means exactly what you suspect: if Astra has discovered that a non-sofic group exists, it is because a human expert first taught it how to “think in the language of math.” AI may already have surpassed them, yes, but without flesh-and-blood math experts, it’d have never achieved these feats. You may see this decision—to willingly participate in training AI at math—as a betrayal to their kin. But remember: not every mathematician regards it as a loss, or as an ending. Some chose math for its beauty; others for its power to uncover truths. Both are valid purposes.

You’re probably wondering now: what’s next? Or rather, why doesn’t this apply to essays or art or screwing in a screw or cooking a Spanish stew—which seem much easier than math to us?

The answer—at least for now—is that those things are not perfectly explainable. There’s an incredible amount of implicit knowledge—the kind that can’t be written down or spoken out—about how to screw in a screw. You can read 100 manuals, but it’s not until you feel the forces in your hand that your brain figures out what “screw in a screw” actually means. If I try to explain why I love this sentence by Marcel Proust—w*e can sometimes find a person again, but we cannot abolish time—*I might turn to its succinctness or the beauty of the themes (time, love, loss) or the precise sting of the words, but it’s really hard to be objective about “why it’s good”—much less so if the creature listening to me has not lived a life.9

Math was made to be perfectly explainable; centuries of cumulative work to build a perfectly self-contained formal system. This effort ultimately failed but produced the best “artificial” language to explain the universe; or rather, to speak with it. Don’t make the terrible mistake of thinking that AI can’t do whatever you do because it’s dumb or because your work is special or because you’re protected by the stars. The only reason AI can’t is that it doesn’t have the right data yet.

Now, in the year of the Lord 2026—neither a prime nor a perfect square—two companies took all of that math data and, with some clever sleight of hand that I won’t go into here, trained a general-purpose algorithm to master the skill. In four years, AI models have gone from being unable to make themselves understood to solving century-old math that you can’t understand.

And now we wait. As the most fervent advocates love to say—and I agree with them on this point—this is the worst AI will ever be. The Riemann hypothesis lies on the horizon, like a final boss.

VI.

And yet—no new math, no new conjectures, no new interesting ideas, no novel premises.

Solving the Riemann hypothesis would be a breakthrough of such a magnitude that it would dwarf everything we’ve seen so far. And yet, solving the Riemann hypothesis doesn’t make one Riemann, for he had to come up with it in the first place. Why is AI better at the former than the latter? Why does it have such strong deductive and inductive skills and yet it lacks in abduction? Why does it tend to find counterexamples rather than proofs?10 Why is it so good now at scouring the abandoned corridors of the cathedral of mathematics, as Bloom calls it, and yet it would have no clue whatsoever about how to build a new wing?

To be clear, I wouldn’t either.

I can’t write a symphony or turn a canvas into a beautiful masterpiece. But some humans can do all of that. Some humans have had a disproportionate influence on building the world around us. They have built cathedral after cathedral after cathedral. What is AI missing?

A DeepMind researcher argued in a Jan 2026 position paper that “LLMs can’t jump,” meaning that they can’t go from followers to trailblazers: “[W]hile a modern Large Language Model could plausibly execute the deductive phase of proving theorems from established premises, it is structurally incapable of the abductive ‘Jump’ required to formulate those premises. We identify the translation of simulation into formal axioms as the critical bottleneck in artificial scientific invention.”

He argues that “physically consistent, multimodal world models offer the necessary sensory grounding to bridge this divide,” which bridges this missing ingredient to the divide I see between how OpenAI and Anthropic approach AI vs Google DeepMind, a topic to which I dedicated a long deep-dive recently. I’m not going to enter again into world-model conceptual territory (read the next footnote for that), but suffice it to say that I find this position paper quite persuasive. (Others call it “goalpost-moving,” so feel free to take the position you consider more reasonable.)11

Coincidentally, it was Google DeepMind’s CEO Demis Hassabis who preemptively reframed the significance of proving and disproving conjectures. In the Big Technology podcast, he said that “Today’s systems are nowhere near [AGI]. Doesn’t matter how many Erdős problems you solve.” I think it’s fair to interpret this as meaning that, in general, solving math problems doesn’t require the same kind of intelligence as posing them in the first place. (One wonders whether he’d say the same thing if it was the Riemann hypothesis that some AI model had solved. We may find out soon-ish.)12

All in all, AI has been solving hard math problems because 1) math is verifiable, and thus it’s not that hard, and 2) problem-solving doesn’t require abductive jumps or world modeling or the ability to come up with new theories, and thus it’s also not that hard.13

So, what’s the big deal again?

VII.

I think the obvious value—besides dunking on goalpost-movers and other denialists of AI’s capabilities—is implicit in the expression “the cathedral of mathematics.”

The word cathedral is in part intended as beautiful and sublime and magnificent but also as big. Mathematics, for those of us who didn’t go past differential calculus, is gigantic. You can spend your entire life going down a single corridor, earn a Fields Medal—one of the highest recognitions in the discipline—and still never find the endpoint nor visit any other corridors in any meaningful way. The cathedral of math is, quite appropriately, as vast as the universe.

There’s immense value—hard to overstate, really—in having at your disposal a tool that can search and dust off secrets in the abandoned corridors of this cathedral. Places no one has stayed for more than a few seconds (in Wikipedia) since the nineteenth century; places that open up doors to dozens of new places; places no one has ever visited.

AI is well-suited for mathematics and even better suited for skimming the vast corpus of human knowledge and finding relics. Think about it. AI knows everything that’s been recorded online (and not so online): From the Big Bang to the eukaryotic cell; how the Great Pyramid of Giza was allegedly built or what are the elements required to make the iPhone in my hand; why international relations are important or how to derive general relativity; the crux in Kant’s critique of pure reason; the value of Cubism and the virtues of capitalism; how the double helix model revealed we’re each unique and how the Heliocentric theory revealed that we are not—it knows about all of that and many more things I can’t possibly know.

The world is a museum of passion projects, and AI has gone to see the human exhibit a million times already. How many connections can it make? How many sentences can it read that have not been read in a hundred years? Human mathematicians could go there, sure. But in a world governed by financial interests and time constraints and reputational risks, the bet is clear: you never go.

If nothing else, the current capabilities of AI models allow us to ponder this: how many more answers we’d have if all it took were asking the question?

One specific insight I’ve gathered from reading the impressions of mathematicians is that, although some of these conjectures are somewhat famous, others are less well-known.

It surprised me to see Tim Gowers, Fields medallist, say that “[the Jacobian conjecture] is for me the first example of an LLM solving a problem not in my area that was nevertheless big enough that I had very definitely heard of it.” My surprise comes because, one, I’d naively expect a master mathematician to know every conjecture that OpenAI and Anthropic find worth reporting on (but math is big!), and two, he doesn’t even seem particularly affected by the Jacobian conjecture itself in the sense that “yeah, I’m somewhat familiar with it” doesn’t strike me as an expression of closenes—or, for what it’s worth, grief.

How many math problems have been forgotten or never tried in the first place because they are hard, long, boring, unknown, unpromising, unfunded, or even career suicide? My bet is many. AI has all the right traits—doesn’t complain, doesn’t tire, doesn’t eat or sleep—to go search those abandoned corridors and explore them farther than humans could justify. Math, unconcerned by the frictions and logistical nuisances of the real world, is the perfect subject to AI’s newfound skills.

And because you, too, have access to AI tools, it means that you—congratulations!—have the makings of a mathematician. That’s another value not to be dismissed.

I know you have understood nothing about the AI’s disproofs I’ve recounted above, but that’s secondary. You can just take AI’s reported capability at face value and become an AI-enhanced mathematician. The AI does the math, and you stamp your name on it.

Or, conversely, you can take that lesson and extrapolate it to areas into which you’re interested. If AI can make these discoveries in math, it’s in part because it’s well-suited for it, but also in part because it just can do this sort of thing. AI is, like a well-trained bloodhound, yearning to go run through the abandoned corridors of whatever edifice you happen to be in. Math is special because it makes clear that AI is trained to do that. Realizing you can apply AI in your work—assuming that, unlike a mathematician or a stew-maker, you have a normal office job—could be the most personal and practical value you extract from this piece.

Another insight I’ve gathered is that AI is weird.

It’s well-suited to pursue unusual routes, ignore standard premises (like assuming the Erdős unit-distance conjecture was likely right and thus devote most time to prove it rather than disprove it), or draw impossible connections between subfields (like using algebraic number theory to disprove the unit-distance conjecture). AI is treating math like it once treated chess and Go, as a virgin arena, untainted by human biases and conventions. Its “alien” intelligence allows it to see beyond.

As writer Kai William says in his review of OpenAI’s unit-distance conjecture disproof, “OpenAI’s solution . . . had two properties that played to the strengths of AI models relative to humans.”

First, he says, Astra brought algebraic number theory all the way over to the unit-distance problem. Math is big, as we’ve seen, and it’d be unusual for a single human to be able to cover an arbitrarily large amount of territory to do these kinds of impossible connections. That’s a task for AI; no matter how immense the cathedral is, AI’s recall and pattern-matching ability are near perfect compared to ours. And second, “the reasoning process was such a grind,” which means that no human in his right mind would have ever gone as far as it was needed to derive the disproof. Williams mentions that Tsimerman had thought of approaching the conjecture as AI did, but that such an approach is, from a human lens, time-consuming and often unfruitful.

I’m reminded, reading Williams’s recount of the story, of how Go masters described the famous “move 37” that AlphaGo played against Go top player Lee Sedol in their 2016 match. They called it a “mistake,” “strange,” “beautiful,” “not a human move.” 2,500 years of humans playing Go, and a machine suddenly invents an inhuman move, one that led to the end of humanity’s rule over the game. Like in chess, no human has beaten the top AI Go player ever again.14

Was this “inhuman move” by Astra in math research a comparable feat? I’m not sure I’d call out “move 37” for math yet. Although mathematics has been around for even longer than Go, there are many things that humans can do in math that AI struggles with. But the date is getting closer—

—especially if all it takes is $2,000.

VIII.

I cited the $2,000 figure from OpenAI’s announcement, but it deserves a second thought: AI is changing the cost structure of intellectual persistence because where a math expert cannot reasonably spend six months trying 400 ugly approaches to a problem that ultimately won’t matter, AI can—on top of retaining the failed routes, cross-referencing remote literature, and returning with either a construction or a better account of the obstruction—do it cheaply. It makes previously irrational research bets—most of them, if you ask a financier—economically rational.

So what will professional mathematicians do now?

Some will just call out anything that AI can do as “not mathematics anymore” and keep going, unfazed. Others will use AI to learn about the branches of mathematics they’re not experts in, becoming a sort of superhuman cyborg. Others will grab the low-hanging fruit, taking advantage of their mathematical knowledge and understanding to prove more and more conjectures.15

But what about the rest, those who don’t welcome AI so enthusiastically?

One possibility is struggle.

When Kirwin Hampshire said, counterfactually, that literary masterpieces would be “dropping as fast as publishers could scoop them up,” he didn’t consider a worse possibility: publishers wouldn’t be able to scoop them up fast enough. There would be masterpieces piling on the floor, waiting for us to review them, incapable of keeping up with AI’s churning rate. What if the AIs scouring the abandoned corridors of the cathedral start to reveal secrets faster than we can digest them?

This has been previously stated as “proof indigestion.”

Consider the implications of à la carte low-cost proof generation. It’s great that one guy can provide a counterexample to an 80-year-old conjecture for as much as 20 bucks; it’s the beginning of a new world we will need to adapt to, but besides our hypothetical PhD math student, it’s hard to see how it could be worse than the alternative (i.e., expensive math). However, such an overhaul of the status quo reveals ups and downs when you inspect the second-order consequences.

Just as correct proofs and disproofs being cheap is a priori good, incorrect ones being cheap is a tragedy. As Terence Tao said in a talk in 2024 on this topic, “normally, when a technology . . . produces something bad, you can tell it’s bad . . . but these [AI] weights are chosen specifically so that the answers resemble correct answers as much as possible, so even when they’re wrong, they look very very convincing.” In a more recent talk—a week before the Astra announcement—he said that “we are entering . . . a crisis in the foundations of mathematical values and practices.” He cited Fields medallist William Thurston’s words on how proofs are not mathematics’ main goal:16

We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about math.

And I add this other quote from Thurston in the same vein:

The product of mathematics is clarity and understanding. Not theorems, by themselves . . . . There is no way to run out of ideas in need of clarification. The question of who is the first person to ever set foot on some square meter of land is really secondary.

Tao closes by saying that “we will transition from an era of proof scarcity to an era of proof abundance,” without solving for all the other required steps to have a complete digestion of a mathematical finding. This is the perfect recipe for AI-manufactured struggle.

This tweet below by a machine learning professor summarizes the idea:

Not only will the poetry of mathematics be lost to AI, as Hampshire writes in his essay, but also the logistical ability to filter brilliance from bullshit or value from excess. As the rate of false positives increases, the toll of human work will shift from discovery to verification, to exposition to digestion; mathematics will face a problem it’s profoundly unprepared for.

Before modern AI, mathematical research being hard and “expensive” (at least in human hours and human cognitive power) was a natural filter for the number of proofs to be verified (and even then, the incentives of academia were pushing this filter to the limit). When one AI-generated proof is submitted, you need to pay close attention; when thousands do, you need a revolution.17

This would be less troubling if Hilbert’s Entscheidungsproblem had turned out to be solvable.

In 1928, David Hilbert asked whether there could be an algorithm that, given a mathematical statement, would determine whether it was valid (”yes”) or not valid (”no”). Turing and Church proved it didn’t exist, shattering Hilbert’s dream, in 1936. AI models nevertheless look like a strange, partial return of that dream. You give it a prompt, it gives you a proof.

What neither Hilbert nor Turing could have imagined was that the machine wouldn’t be built but grown, like flowers in a garden. AI’s unreliability is so consequential in mathematical research because the same system that can extend the frontier by proving non-sofic groups exist can also manufacture errors with enough fluency to survive serious inspection. The implications of the machine that we ultimately created run as much in the bad direction as they do in the good one.

All in all, we have that 1) AI will produce too many potential proofs and disproofs for professionals to handle, 2) the scientific conditions around these findings are at best imperfect and at worst missing, 3) proofs themselves are not all that important to the purpose of math without understanding, and 4) AI will make a lot of mistakes that look like “plausible but ultimately wrong solutions.”

The combined impact of both these avalanches will saturate whatever bulwarks are still standing on the human side of mathematical research. How can the mathematics community digest the situation? Hardly so.

That’s why a group of mathematicians from Leiden University, on behalf of the global community and with extensive feedback from it as well as an endorsement from the International Mathematical Union, Terence Tao, and many other distinguished mathematicians, published a Declaration on Artificial Intelligence and Mathematics. One of their points is precisely that also understanding, and not just a body of results, is what mathematics is about:

Mathematics produces not only a body of results, but also understanding, clarity, and judgment among the communities of mathematicians who have shaped them, often in the context of their own autonomously guided research. This expert knowledge is essential, both to effectively use mathematics, and to continue to articulate new and significant research questions. A key source of strength of the discipline has long been the autonomous shaping of the direction of research and the methods used to pursue it.

That said, they can always pivot their career to cracking math jokes online that only math people get.

IX.

Another possibility is to move on.

While moving up the ladder of abstraction to play the role of “connective tissue” is not available to everyone but a handful of math geniuses, moving on is available to all.

It’s a terrible thing to be pushed out of your dream life, but rather than remain in denial, angry, or depressed, it’s better to just find something else. Like a job at OpenAI or Anthropic. I’m kidding—partially. How many mathematicians do you think have worked to make these milestones possible? A damn lot. So you can be one of them. Or you can go play outside for a while and leave behind a “moribund academia.” And then find a job in a non-verifiable field, if possible, like sociology.

Tsimerman says we should prepare for math not to be “viable as a professional endeavour” anymore:

I think problem-solving is an immense, and pervasive part of modern mathematical research. Consequently, if human problem-solving disappears by virtue of the AIs becoming strictly and substantially better at it, then most of the time currently spent by modern mathematical researchers will have to be spent on an activity that is altogether pretty different. Whether such an activity is viable as a professional endeavour is something I am unsure of, but strongly encourage others to think about and try to envision, so that if/when the time comes, we can steer such a future into being.

I hope this doesn’t happen, but math might be hollowed out due to both the shrinking career prospects and the perceived status loss.

Appearances matter as much as truth does to young people risking their careers on an imperfect-information bet. What aura is left for a field that’s been meme’d on with “hey there the Jacobian conjecture is false”? Non-mathematicians are calling out “the end of math” and “the most significant single day in the history of mathematics” at the same time. Being squarely opposite in meaning, these assertions reveal an equal amount of contempt and ignorance: they care so little about math that this looks to them both like the first and the last day of the field (death or rebirth). But it doesn’t matter if they’re wrong and math is fine: to be willing to choose math, a young student has to witness it from the outside before experiencing it from the inside; I wouldn’t bargain with my future like that from what I’m seeing right now. Math is not software development.18

Kids that would’ve otherwise been future brilliant mathematicians may as well just move on. If AI will nevertheless get all the credit in the eyes of the world, why bother?

Do you think that AI taking the credit is an outlandish possibility? Here’s what the Leiden Declaration has to say about that:

Technologies that draw extensively on the published mathematical commons undermine the traditional system of attribution. Models trained on published works frequently return outputs that do not properly cite the human works they synthesize. Many current models are also built on data obtained by systematically exploiting licenses and access arrangements that were not made with artificial intelligence in mind, or indeed by simply violating copyright protections

If a model finds a half-worked proof that was discarded, who deserves attribution for the correct result? AI can explore as many abandoned corridors as it wants, but those corridors were once explored by humans. A human would know how to credit, but AI’s sourcing is often sloppy. OpenAI actually responded to this point by the Leiden Declaration on the Astra blog post:

We believe attribution should honestly reflect how a result was produced: claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system’s contribution and the nature of genuine human intellectual work. We helped prepare the manuscripts and formalize the proofs in Lean, and we take responsibility for their correctness, while the mathematical arguments themselves were generated by our system.

I will note explicitly the irony here: it’s inspiration when AI is trained on the entire corpus of human knowledge but misrepresentation if a human claims authorship for a proof generated with the help of an AI tool. Even if we take this seriously, then whose attribution is it? The AI? The company? Will OpenAI train its tools to ensure proper attribution and sourcing of the literature? Will they make sure to clarify how much of a generated proof is extended or resurfaced from existing work? I don’t think they are in a position to make such a petition.

As I was finishing this piece, I found an interesting paper published on August 4th. Group theory mathematician Francesco Fournier-Facio qualifies OpenAI’s discovery of a non-sofic group, citing previous work, saying that it invalidates “the claim in the OpenAI announcement on 1 August that there has been ‘no progress on the main result for at least a decade.’” OpenAI eventually redacted the claim, but it took one day for someone to find a clear case of bad sourcing and misattribution.19

I understand why some mathematicians are extremely happy.

I also understand why some mathematicians are mad and sad.

X.

I find it impossible to accept any reduction of this news to either “this is hell on Earth” or “so good, the Singularity is here.” There’s so much nuance to be disentangled and debated and understood that those comments respond more to the speaker’s emotional reaction than to any sober reflection. (This piece took 10,000 words and I left out so much I’m not doing justice to the story.)

I end with this.

We may be closing the distance between ourselves and the future, but the oldest advice still holds. Technology might alter the face of the world beyond recognition, but it cannot spare us our human condition. OpenAI, in an attempt at removing the difficulty of being human, called its model Astra, but I put us back where we belong: Per aspera, ad astra. Through hardship, to the stars.

6

I interpret Tao’s “understands” as referring to the often gibberish output of AI models. I don’t think he’d say he can’t understand one such proof due to its length or complexity. So this note is to clarify that I’m pushing his meaning of “understands” further in my assertion: what if, even once the proof is well-written, almost no one can grasp the math? I imagine that me saying this about Tao sounds preposterous, but consider that Wait But Why chart where the distance between a chimp and a smart human is nothing compared to a superintelligence. At some point, if AI keeps getting better and better, it will go deeper and further than any human can follow—invent new math, new concepts, whatever—even Tao. That is a narrow version of a superintelligence (a mathmaster, if you will). And I contend that that sort of superhuman research wouldn’t have ~any effect in the world because it needs humans playing the role of connective tissue to bridge the gap between knowledge and reality. A proof needs to be generated, then verified, then presented, and finally, understood and applied.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

This is a long deep dive (~8,000 words) with a lot of information and many threads to pull on if you're interested. I’d appreciate it if you support this kind of deep research that takes more time and effort than usual.

What if I told you that Google is out of the AI race?

It didn’t happen because Google lost, though. Despite an underwhelming I/O event earlier this year, several heavyweight departures, and a slower model release pace than its competitors, Google is still alive. It is out of the race because it withdrew. Weird, right? Why would Google do that willingly?

To answer that, I’m going to tell you the story of why and how Google left the AI race that OpenAI and Anthropic are betting everything on.

Here’s my hypothesis, stated plainly: Google DeepMind CEO Demis Hassabis—a pioneer of AI as we understand it today—doesn’t think that automating AI research with coding agents (AI systems that can program better AI systems) is the correct approach to artificial general intelligence (AGI, the kind of AI that’s as good as humans at everything). Google DeepMind’s leadership sees OpenAI and Anthropic’s bet at best as an off-ramp, and at worst as a dead end.

Hassabis is betting on something else: world models. Models that can understand and simulate the real world, not just predict the next token.

And he’s steering Google accordingly. The AI race is actually a race between two theories of intelligence and also between two kinds of company: startups that need AI to become a business, and an incumbent whose existing business can subsidize AI for years. As we will see, Hassabis’s decision puts Google in a life-or-death situation. He could make Google the absolute leader or an irrelevant laggard.

That said, to write Google off the story would be a serious mistake. I’m going to explain why I think Google has a unique chance here.

You won’t see official confirmation about my hypothesis; as far as the executive is concerned, Google is still competing directly with OpenAI and Anthropic. I contend that’s not quite true anymore. Follow the breadcrumbs with me. I’ll put them together into a coherent picture. The evidence is public, but that doesn’t mean I’m right. This is informed speculation, so please, take it with a grain of salt.

If I manage to change your view of what the hell Google is doing, then I’ll be satisfied with my work.

I. AS ANTHROPIC AND OPENAI RUSH FORWARD

The first piece of the puzzle is that Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman think that AI models will soon be autonomously improving themselves—this is known as “recursive self-improvement” (RSI): model A makes a better model B that makes a better model C, etc.—and that this is the key to achieving AGI.

The best approach to pursue this goal is also the simplest: build datacenters and use brute force on algorithms that can be scaled that way, like learning and search. That’s the fundamental insight of Richard Sutton’s Bitter Lesson, which both OpenAI and Anthropic hold sacred. If new architectures or algorithms are needed, those better models, equipped with a lot of compute, will find them somehow.

That’s the theory. What does this creed look like inside the lab? We only need to read their testimonies.

You have engineering staff at both companies admitting that they barely write any code anymore. It is swarms of coding agents—Claude Code and Codex—that do. Employees have become managers of agents rather than developers and researchers. Claude Code writes Claude Code and wrote Claude Cowork. GPT-5.6 Sol “autonomously post-trained” GPT-5.6 Luna. They say straight away that “RSI is coming.” Human supervision is still required for many things—we’re in the final moments of the “human-in-the-loop” stage—but one prediction is that by 2028, there’s a 60% chance that RSI is achieved. That’s what Anthropic co-founder Jack Clark wrote in a newsletter in May. Others predict RSI even sooner. Things are going so fast that Anthropic is in favor of a coordinated slowdown: it’d be “a good thing.” OpenAI staffers, even those who advocate for cautious progress over excessive fear-mongering, are saying things like: “if we could coordinate a global capabilities slowdown today i would likely press that magic button.”

So what we have here are the two top AI startups deeply convinced on one goal and one trajectory and pushing everything to get there. If I were Google, I’d have to be 100% convinced this isn’t the way. The rate of progress displayed by Anthropic and OpenAI is vertiginous for the world and rather unsettling for their competitors.

Another AI pioneer, Andrej Karpathy’s decision to join Anthropic earlier this quarter as a “member of technical staff” surprised everyone. “What did Karpathy see?” people asked. Well, he saw the exact same trajectory toward RSI that the labs were seeing. He joined a team tasked with teaching Claude to improve the pre-training phase of the next Claude version. This basically means: making AI’s foundations more robust so that the AI can make the next AI’s foundations more robust, and so on.

Karpathy knew RSI was possible. He had built a miniature version of RSI with his autoresearch project himself: he discovered that AI agents could autonomously improve the efficiency of his own code by ~10%. If only this could scale, he wrote, then they’d step into the next stage. Anthropic was a good place to try.

And so he joined Anthropic, a lab with a conviction bordering on religious devotion toward the scaling laws derived from Sutton’s bitter lesson. But what else do you do when you stand on the verge of the exponential, right? You can only embrace it, like one embraces the sublime of a huge mountain or an inexorable hurricane.

Being six years younger than OpenAI, Anthropic had one shot, and they hit the bull’s eye: An unbreakable focus on code + agents + enterprise secured a steady climb to the top. It’s been a crazy streak of successes, from revenue to valuation to popularity to getting ahead of larger and older companies, passing through several back-and-forths with the US government.

OpenAI is a different story.

It skyrocketed to fame with ChatGPT, and then it had to deal with the success disaster that followed: too many expensive free users not willing to pay. At the time of writing, the revealed preferences of US households indicate that only 2.2% are willing to pay for an AI subscription. Despite that, OpenAI is not behind Anthropic in any relevant metric: they’re essentially tied on revenue (run-rate revenue looks better for Anthropic, but it’s not the same as actual revenue), reach, and ambition.

Perhaps most importantly, OpenAI fixed its main weakness: over-diversification. Earlier this year, Altman cut secondary projects that were taking up invaluable compute resources and distracting from the path of RSI. As I wrote back then:

OpenAI spent 2025 announcing Sora, a video generator, Atlas, a web browser, a new device, and e-commerce features for ChatGPT. Altman compared this to “betting on a series of startups.” I called it “risky.” In March 2026, OpenAI’s CEO of applications, Fidji Simo, told staff they were in “code red.” (A second at that.) She described Anthropic’s enterprise gains as “a wake-up call” and cancelled all “side quests.” Current and former employees described a period of strategic confusion: mixed teams, unstable compute allocation, a cascade of restructuring decisions...

This move by OpenAI was read by analysts as a push to catch up to Anthropic’s unmatched revenue figures on the enterprise market, but my reading puts revenue in a secondary place. OpenAI will not worry about Anthropic having more revenue than them at any given point—they worry, if anything, about the trajectory.

OpenAI realized that Anthropic was starting to show the signs of acceleration consistent with proto-RSI. They care about money, sure, but Altman wouldn’t make a company-wide restructuring for a short-term trade. This was a full overhaul intended to be a long-term investment. OpenAI can afford to let Anthropic win the enterprise market, but it can’t afford someone else achieving RSI-enabled escape velocity.

OpenAI pivoted fast. It doubled down on Codex, agents, enterprise, and, ultimately, on the road to recursive self-improvement. OpenAI is doing so well now that insiders confidently claim that Codex is actually better than Claude Code. Codex actually grew well beyond expectations after the release of ChatGPT Work, and it’s up to 10 million users, as Bloomberg last reported.

Anthropic’s Fable 5 is great but too expensive—as is Opus 5, just introduced—so in terms of cost-performance ratio, which is the most interesting metric right now, GPT-5.6 Sol seems to be superior. (If you’re wondering what model to use for which task, I wrote a simple guide for beginners.)

But then next month, people will confidently claim the exact opposite. “We’re so back”; “it’s so over.” Etc. Etc. So who’s winning? I don’t know, I’m not an oracle; I’m just a breadcrumb follower. And the bigger picture reveals that it doesn’t matter whether OpenAI or Anthropic is better off, because during the exponential climb, anything can happen, and what counts is not so much the point as the trajectory.

What I know is that those left out of the first stage of the exponential are out of the game. And as far as we can tell from public information, that would be Google.

Anthropic made this strong claim far back, in April 2023: “We believe that companies that train the best 2025/26 models will be too far ahead for anyone to catch up in subsequent cycles.” This is what I mean when I say “escape velocity.” It’s the speed you need to escape the gravity field of the planet without falling back to the ground. Once you get there, your next model release is accelerated thanks to the previous model’s involvement in training and post-training. At the time of writing, everything Anthropic and OpenAI argued in 2023-2024 has become true.

All of the above happened—gradually then suddenly—in the first few months of 2026. Where’s Google in this entire story? Can you see them? Yeah, they have Gemini 3 point something. And Antigravity. And something called Omni—more on that later—but they’re essentially nowhere to be found at the frontier of the code + agents + enterprise + RSI race. Google released its latest model—Gemini Flash 3.6—on July 21, 2026. Here’s where it landed in the Artificial Analysis intelligence index:

That is an unacceptable score for Google. Something very big must be happening inside the search giant for it to be giving up the good press and the extra revenue of achieving RSI. It can be as big as the corporation falling apart or that it’s setting its sights on a bigger prize.

What can be so big…?

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

My take on the current situation between America and China on AI.

Some news on Moonshot’s K3: the White House has concluded that Moonshot AI, the company that’s achieved notoriety after the release of Kimi K3, used a sophisticated attack to exploit the superior performance of Anthropic’s Fable.

This is a serious offense that could prompt measures to restrict Chinese AI in some way. Here’s the announcement, published on X by Michael Kratsios, Office of Science and Technology Policy director, presented without any evidence whatsoever:

We have information that Moonshot AI distilled Anthropic’s Fable for the development of its K3 model.

To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection. Moonshot AI has also acquired GB300-equipped servers and has accessed GB300s in Thailand, likely to train its AI models.

The United States strongly supports the free and fair development of AI, including a thriving competitive ecosystem that spans frontier models, specialized systems, open-source frameworks, and open-weight models. Legitimate AI distillation used to create smaller, more efficient models plays a vital role in this open innovation ecosystem. However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.

And here’s Treasury Secretary Scott Bessent:

We support open-source AI and the innovation it unlocks. But open source is not open season on American IP. When PRC firms conduct covert, industrial-scale distillation attacks that cross the line into IP theft, sanctions and Entity List designations will be on the table.

If the accusation of distillation and IP theft are true, this means that Moonshot needed American models to exist in the first place to achieve such impressive performance with K3. In other words, China can ride on America’s slipstream but perhaps not make its own.1

Many people dismissed the possibility of a distillation attack right after learning about Kimi K3’s benchmark scores. It was superior to Fable in a few of them, which indicates genuine prowess. After all, you can’t distill a teacher model into a better student model. I saw merit in this view, but I was careful to take a more nuanced stance: maybe distillation did take place, and, on top of that, Moonshot is a company of great talent and hard-working people.

Those two things aren’t in conflict.

The one who’s in conflict (of interest) is Anthropic.

It has all the reasons to accuse Chinese AI efforts of being strictly derivative of their IP. Anthropic’s ties with the US government ensure a channel for lobbying against open-source AI, whether Chinese or national. After all, Anthropic’s revenue is threatened by any kind of open initiative—you can’t sell subscriptions if there’s a free, open, cheaper option waiting one click away. The difference is that domestic open-source initiatives have been choked under the increased closeness of the biggest AI labs and the negligence of regulators. So China is what’s left.

Both the Chinese government and the Chinese industry have stated in clear terms that open-sourcing AI will remain a central goal. The trade-off between openness and closeness exists for them as well, but it is different from the one facing an American model company like Anthropic or OpenAI. China can accept lower direct revenue from model access if open models spread Chinese technical standards, strengthen domestic industry, and become the default option in countries where American systems are expensive, restricted, or unavailable.

Open source can support a viable business. DeepSeek’s approach, for instance, is to give away the weights while charging for efficient hosted inference through its API. Moonshot does the same thing. And so does Zhipu. China is genuinely committed to open-source AI. Most companies do not want to buy servers, optimize kernels, manage capacity, and maintain their own deployment. If these startups can run the same model more cheaply than customers or rival hosts, it can earn a reasonable infrastructure margin—even if not the kind that a software monopoly would enjoy, which is what OpenAI and Anthropic are aiming for—while still allowing broad adoption. The geopolitical benefit and the company’s financial return are separate, but they reinforce each other. The world using Chinese AI because American AI is expensive, or close, or capped, or lobotomized, or safeguarded, or not allowed overseas is a total civilizational victory.

There’s another reason not to fully believe Anthropic or the US government on this: Fable was not available during most of K3’s training and post-training.

The dates just don’t match. Fable was released to users—API and subscription—on June 9. It was taken down three days later at the request of the US government due to alleged cybersecurity risks and didn’t come back online until June 30. Kimi K3 was released on July 16. Two weeks is a negligible amount of time in AI training timescales. K3 certainly finished training much earlier than Fable was even made available. There’s a possibility that Kimi K3 did some quick post-training on it on top of their own post-training, but I doubt it was enough time to extract much data from it.2

There are some alternative explanations, with some gaps each.

First, Fable is a version of Mythos “made safe for general use.” Anthropic announced Mythos on April 7 (never released publicly) and made it available to a bunch of national partners through Project Glasswing, including the US government and technology companies. This was presumably made with the greatest care given the alleged threat level that Mythos embodied, were it to ever fall into foreign hands. Could Moonshot have hacked onto Mythos through one of the partners? Maybe. That would be really bad. But also unlikely, given 1) nothing terrible has happened to essential American infrastructure that we know of which might have happened had China gotten access to Mythos, 2) the White House has explicitly said “Fable” was distilled, not “Mythos.”3

Another possibility is that Moonshot was actually distilled from Opus-class models—I think this is much more likely to have happened—and the story got accidentally tangled. This would mean distillation did take place and thus the accusations have some basis. But it would not explain why K3 is so good, which is actually the bigger problem for Anthropic—just one against which they have no claim. What if Moonshot did some distillation on less powerful models and then found ways to make K3 as good as Fable? That would be especially bad news for Anthropic—and its investors—because it would mean Chinese companies can close the performance gap by distilling from the second-best models, without needing access to Anthropic’s frontier model at all.

Whatever the case, I feel there are two key readings on this whole situation. I’m much less concerned about whether Moonshot did or did not distill Fable than about how Anthropic reacted.

First, if Anthropic can’t defend itself from a smaller, weaker AI lab using Fable, how do they expect us to believe AGI is near? If Anthropic can’t protect its lead because distillation is impossible to stop, then how can they argue in favor of keeping the models closed in the name of general interest? If China will always steal the gains and release them into the world, then Anthropic should do it first. It’s not just that open-source is the best strategy for America, but perhaps the only one.

And second, how does Anthropic dare make accusations of “IP theft”?

In June 2025, a federal judge ruled that Anthropic had violated authors’ copyrights by downloading and storing more than seven million pirated books, although its use of books to train its AI was deemed fair use. In July 2026, another judge approved a $1.5 billion settlement to compensate affected authors and publishers.

If we translate this to the Moonshot story, it means that using Anthropic’s IP “to train its AI” should be “deemed fair use,” unless it was being stored somewhere. Why is it reasonable to deem the practice of taking the entire web—that humans collectively create—as data to sell it back to us, compressed in app form, but giving back that compressed data to us is suddenly an “industrial distillation” attack? As I see it, Moonshot AI is the industry’s Robin Hood: it steals from the rich—who previously stole from us—and gives to the poor.

But I want to be measured and charitable with Anthropic, as I was with Moonshot before: maybe a heist of the entire internet did take place and, on top of that, Anthropic is a company of great talent and hard-working people.

Those two things, I say again, aren’t in conflict.

Unfortunately for Anthropic, it doesn’t have the last say on this topic.

Wired reports that the “Trump admin is divided on whether to restrict Chinese AI models,” citing familiar sources. The White House doesn’t want distillation at large scale to happen, “[b]ut an executive order to ban Chinese labs from accessing US models is not currently on the table. Meanwhile, Commerce instead wants to find ways to incentivize US companies to develop open models to counter China.” Politico reports that the tech industry is also doubtful that a ban on open-source Chinese AI is a good idea: “Nearly 200 Silicon Valley companies, including Proton and Y-Combinator, are urging the Trump administration not to cut off access to Chinese open-weight AI models or risk crippling the next generation of U.S. startups.” Here’s a chart on the divide:

To Anthropic’s surprise, not everyone seems to have Anthropic’s best interests at heart.

But even if the entire government and the entire tech industry were to side with Anthropic—which would be a terrible strategic mistake for America given the potential tipping effect of Chinese AI spreading through the world—the outcome will ultimately be decided by what people actually do. The primary force in this affair is no Bessent or Amodei, but whether enterprises and individual consumers are okay with using Kimi K3 and DeepSeek V4 and GLM 5.2, MiniMax-M3, etc.

If data is to be trusted, they certainly are:

Here’s what Bloomberg says:

Chinese models have already gained traction in the US, making restrictions harder to impose without disrupting American users and businesses. On the AI marketplace OpenRouter, they account for nearly 60% of token usage by US companies. Silicon Valley startups and researchers rely on customizable Chinese models, while DoorDash Inc. and Airbnb Inc. have adopted them as low-cost alternatives or complements to American offerings from OpenAI and Anthropic PBC.

Bloomberg reports that, in the face of these haunting numbers, Bessent “suggested pressure could be placed on companies using Chinese AI.” That is… a way to approach this situation. His position amounts to an admission: “China is winning, so our only option is to punish those who rationally side with the winner.” Not quite the best look for the world’s foremost “defender” of free markets.

Another approach would be this: “let’s outcompete China by pushing open-source initiatives.” That is the alternative Commerce supports, the one I think has the best odds, and the one Anthropic cannot afford to see succeed. Indeed, the only people unlikely to see it as eminently sensible are, first, those more fearful of China than confident in America, and second, those who stand to lose regardless of whether the open-source AI that prevails is Chinese or American—because, despite their repeated geopolitical framing, their real objection is economic.

That is, Anthropic and OpenAI.

OpenAI is part of the broader push against open-source AI. Sam Altman’s team has taken a few tentative steps in the open direction, as the SemiAnalysis chart below shows, but its underlying position is pretty clear. Anthropic may be more overtly hawkish on China, but Altman has previously written he prefers the future to be shaped by democratic institutions rather than authoritarian ones. Neither company wants China to lead.

The obvious question is why are they so afraid of seeing China win when the leading American models remain far more capable?

Consider a recent cybersecurity test. An undisclosed OpenAI model—rumored to be GPT-6—bypassed the safeguards built into the evaluation and hacked Hugging Face, an American open-source AI company, simply to complete a task (it was just following instructions, albeit rather unorthodoxly). How is that not already AGI, right? Hugging Face detected the intrusion and traced it back to an American lab, which turned out to be OpenAI. Everyone involved was quite spooked about the incident. But the episode ended in a partnership between the two companies rather than, say, public recrimination, a degree of tolerance that a Chinese startup would be unlikely to receive.

When Hugging Face tried to find out what had happened, its available OpenAI and Anthropic systems—Fable and GPT-5.6 Sol—were so heavily restricted that they refused to assist. The American company had to turn to a Chinese model after the attack of an American one. It’s not often that reality hands us the correct answer so straightforwardly.

What I find most surprising about all this is that people are genuinely afraid that a model might break out of its sandbox environment—because that would demonstrate its misalignment or something—rather than fearing the real problem: if America doesn’t change the way it does things, it will be handing China that bright AI future on a silver platter.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

This is a follow-up post to the one I published on Saturday about Moonshot’s Kimi K3.

Moonshot released Kimi K3 last week. It’s an awesome model, on paper as good as Fable and GPT-5.6 Sol. And it’s open. And it’s Chinese.

I wrote a deep dive to answer three questions:

• How good is Kimi K3?

• How is Kimi K3 so good?

• How does Kimi K3 being so good affect the American AI industry?

You can read it here.

But many questions are left unanswered. What will happen next between China and America? What are the second-order consequences of China having caught up to America?

Note that it’s not just Kimi. It’s also DeepSeek. And Qwen. And GLM. China is genuinely catching up. And so the AI race has changed. And so the world has changed.

Here’s the outline for today’s post, where I give you 7 profound consequences of these events:

1. FRONTIER CAPABILITY BECOMES HARDER TO MONETIZE

2. OPEN-SOURCE AI GAINS CREDIBILITY AMONG ENTERPRISES

3. EXPORT CONTROLS LOOK LESS EFFECTIVE (NOT USELESS)

4. THE TRILLION-DOLLAR AI BUILDOUT IS HARDER TO JUSTIFY

5. NVIDIA CAN WIN WHILE AI LABS LOSE

6. BUSINESS VS GEOPOLITICS: WHAT HELPS AMERICA HURTS AMERICA

7. THE POINT MATTERS LESS THAN THE TRAJECTORY

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

I’m on vacation this week but made time to write this deep-dive on Moonshot’s Kimi K3, the first Chinese AI model at the level of America’s frontier models. And it’s open.

Moonshot’s Kimi K3 is the first Chinese open-source model to reach the level of the best American frontier models. In some areas, it’s on par with Anthropic’s Mythos/Fable and OpenAI’s GPT-5.6. For those of you who remember DeepSeek’s ascent to notoriety in January 2025, this is Moonshot’s “DeepSeek moment,” except more serious: Experts used to calculate that China was only six to nine months behind the top Western AI models.

Well, make it zero months, zero weeks, and zero days now.1

DeepSeek proved that China can make an open model much more efficient than American AI labs without a large performance penalty even under severe hardware restrictions. Moonshot builds on that to prove that China can exploit those efficiency-gains-under-constraints to make an open model at the intelligence level of the best American AI labs. Expect, as the main consequence of this, for the geopolitical discourse around AI to increase in both intensity and urgency (at least after Moonshot publishes the model weights on July 27th). If the US government considered Mythos dangerous enough to withdraw from Western allies, what will it think about China having a Mythos-level equivalent? A bad regulatory move could push the entire world ahead of the US.

But before getting into the interesting analysis, let’s do a quick objective review of the model’s capabilities and its relative position inside the frontier AI ecosystem: Why is it making a fuss? Is this a new “DeepSeek moment”? Is it really as good as the best American models in general or only on less relevant benchmarks? All these questions can be summed up with this one: How good is Kimi K3?

I. HOW GOOD IS KIMI K3?

The first place we have to look is, naturally, the self-reported performance scores.

Capability-wise, K3 lands first, second, or third across benchmarks and displays a superior score-cost ratio across coding and agentic evals. It is essentially one rung above Opus-4.8 and GPT-5.5, and almost on par with Mythos/Fable and GPT-5.6. This means that one mistake by Anthropic or OpenAI will lead China to become the main AI superpower in the world (barring a clear inferiority in terms of chips and fabs, which is the hardware basis of AI models; more on that later).

This is what Moonshot itself says, though, which has an obvious incentive to pump its numbers. However, independent analysts agree for the most part. It’s not just Moonshot glazing its own model but the general sentiment. Here’s a short compilation of Kimi K3 being really good.

Artificial Analysis, a benchmarking firm, on how Kimi K3 compares to Anthropic and OpenAI’s top models (blue, brown, and black respectively):

The jump in capabilities and intelligence from Kimi K2.6 is huge. K3 falls only slightly behind Fable and GPT-5.6, both released recently—and at the price range of GPT (Claude is more expensive): “Cost per task ($0.94) is similar to GPT-5.6 Sol ($1.04), ~1/2 the price of Opus 4.8 ($1.80).” Kimi K3 is an appealing model for API users according to Artificial Analysis’ indices (it’s the only model in the frontier of capabilities that falls inside Artificial Analysis “most attractive quadrant”).

All in all, K3 will be an interesting compromise in the enterprise sector, as it lies comfortably between the cheapness of worse open models and the expense of closed ones at around its level.

Arena, another benchmarking firm, reports that Kimi K3 is considerably better than Fable on frontend coding. This was the first result to go viral on K3’s capabilities, but I think it was extremely over-hyped in the sense that one benchmark on Arena means essentially nothing. It adds to all the other benchmarks that put K3 at a really, really good level—it’s just that “much better than Fable” is an unrealistic exaggeration. (The x-axis on the chart below that went viral is truncated at 1,450 to make the difference seem larger.)

I mean, we can be excited about the technological competition or worried about China “getting ahead,” but once you take a look at the amount of green in the chart below on “US vs China advantage,” you will calm down. Let’s not let excitement blur our judgment into a bad analysis.

Indeed, K3 falls considerably lower on Arena’s overall text ranking.

More great results from K3 here. It is also #1 on creative writing, #1 on Vercel’s Next.js agentic benchmark, and #3 on deepSWE, a long-horizon software engineering benchmark.

The overall impression I get from the benchmarks—as well as the reviews I’ve read, which, although anecdotal, add to the vibes—is that Moonshot’s portrayal of the model is accurate. It’s evidently a frontier model; there’s no denying that, and China has caught up to the US, but the top labs—Anthropic and OpenAI—remain slightly ahead. The bottom line is that there’s no comforting gap between the frontier and China for American labs to enjoy anymore.2

The question of “how good is Kimi K3?” which we have just answered with a resounding “really good,” leads us directly into our second question:

How is Kimi K3 so good?

II. HOW IS KIMI K3 SO GOOD?

This is the million-dollar question. How on Earth did a smaller, resource-constrained Chinese open-source lab get to compete toe-to-toe with the extremely wealthy, extremely resourceful, extremely-spoiled-by-the-government American labs? To answer what seems to be a much harder question and also a more unexpected one, we need to consider several factors: size, training efficiency, algorithmic advances, and hardware limitations. There are also some far less pleasant possibilities: when it comes to Chinese AI, unlike in the US, everyone is guilty until proven innocent. Has Moonshot distilled from existing American models? Have they gamed the benchmarks or benchmaxxed the models? Have industry secrets leaked or diffused?

Let’s take a look.

A sneak peek at the positive response, which amounts to “Moonshot, like DeepSeek before them, seem clearly cut from a different cloth,” can be summed up this way: constraints breed creativity. Contrary to what it might seem, K3 is great because of Moonshot’s limitations as a small Chinese lab rather than despite them.

There are plenty of technical details in the blog post on Kimi K3 (thank you, open source). I don’t want to bother you with too much jargon, but a bit of it is always necessary to understand what’s going on. The first detail, and a rather surprising one, is that K3 is huge. The habit of using parameter count as a proxy for model capability faded away with the increasing closeness of the top labs in recent years. (Do you remember the GPT-4 circle vs GPT-3 circle viral image?) But open models don’t have time for secrecy, only progress and transparency. So we know K3 is 2.8 trillion parameters. According to Moonshot, it’s the first open model in the world in the 3T range. (Top models from OpenAI and Anthropic are thought to be in that or slightly higher range.)

Here’s a chart of the size increases of open models over time and a comparison between K3 and OpenAI’s open-source model (everyone went crazy that OpenAI would open a model, which is in itself funny when written down, and it gets only funnier when you look at the size comparison):

One reason not to get too excited about the number itself is that size is and isn’t a proxy for capability. You can have a smaller model that’s better than a bigger one, no problem. But, if done well, a large pre-train will naturally be better than a small one. So, although it’s a significant factor in capability, depending on who says “mine is bigger,” you can or can’t trust it to be a reliable heuristic for quality. (I won’t say here who I trust and who I don’t, but my reliability ranking is quite… broad.)

Another reason is sparsity.

Kimi K3 is extremely sparse, meaning that even if there are 2.8 trillion parameters in the model, the active ones at any given time are a tiny fraction of that. To be precise, K3 is a mixture of experts with almost 900 experts (specialized groups of parameters that are called on only when they are useful for the current task). Only 16 are active at a time, which means the model can draw on a very large pool of capabilities without using all of them for every request.

Combined with additional custom changes to how data moves through the model (Kimi Delta Attention and Attention Residuals—you don’t need to know what those are except in the sense that they enable two-digit efficiency gains at negligible cost increases), as well as improvements on the training process (INT4-native quantization), inference (expert parallelism) and data quality, Kimi K3 is roughly 2.5 times more scale-efficient—efficient at turning energy into intelligence—than Kimi K2. In short: Moonshot engineers are masters of turning scraps into gold.

In case you’re interested, here’s a snapshot of Kimi K3’s architecture:

There’s one common thread that should not go unnoticed here. Moonshot and DeepSeek are alike to the same degree that American labs are to one another, but not for the same reasons.

The reason why Anthropic and OpenAI have pursued the same goals (e.g., recursive self-improvement) and keep a similar pace of progress despite algorithmic advances and efficiency improvements often being unplannable eureka moments is that AI talent moves around a lot in the Bay Area. They all know each other and, naturally, the secrets of one lab are passed to the other without even the possibility of enforcing NDAs or IP claims. The trade secrets of the AI industry are kept from the world but not from one another.

However, the reason Moonshot and DeepSeek are alike is that they’re born out of the same ecosystem—an open and severely constrained ecosystem that needs to grow around scarcity, not abundance. They’re heavily incentivized by hardware shortages to keep everything in the open to benefit from other labs’ discoveries and grow together as well as to keep a razor-sharp focus on finding alternative paths that account for resource limits. This means that they don’t need to share DNA as OpenAI and Anthropic do so much as share a vision: let’s do more with less.

The idea that “constraints breed creativity” is mostly a slogan, but it’s also been seriously studied. A cross-disciplinary integrative review found that creativity responds to a U-shaped relationship with constraints. Too few and you will wander. Too many and you will stall. As I see it, OpenAI and Anthropic are in one extreme of the U, being too wealthy to be forced to bother much about efficiency and stuff except when business pressures ask for it (instead, they will reset limits as if that’s equivalent to making the model as cheap as it can be). xAI is perhaps the clearest example of this in the American AI ecosystem: huge datacenters, zero focus.

Chinese startups inhabit, instead, the middle of the U: constrained hardware-wise but capable of compensating for it with technical prowess and incredibly talented teams that will never scale a model by brute force but by ensuring first that the corresponding efficiency gains are in place. (Don’t get me wrong: American labs also have incredibly talented teams, but they might not be as motivated to direct that talent fully to overcome non-existent constraints.)

As a Google DeepMind researcher, Anika, said: Moonshot demonstrates that “training is efficiency-compressible . . . a small lab with taste can compress the compute needed to make a frontier model, even if it can’t afford to serve one. the frontier is no longer something money can buy.”

But being creative also means hacks.

There are many cheap tricks a small AI team can deploy to benefit from the closed advances. This alternative path to greatness leads me to reassess Kimi K3’s capabilities: are all these advances on efficiency and intelligence legit, or has China copied America once again?

This has been the usual explanation for the question of “how has China caught up to American AI efforts so quickly?” Anthropic CEO Dario Amodei has stated Anthropic has detected several “industrial-scale campaigns” from Chinese actors—including but not limited to Moonshot—distilling Claude models. Although Amodei’s hawkishness against China has often been the target of mockery, this is not a minor issue. Quoting from Anthropic’s blog post in February:

Distillation attacks undermine [chip export] controls by allowing foreign labs, including those subject to the control of the Chinese Communist Party, to close the competitive advantage that export controls are designed to preserve through other means.

The first problem is of a business nature. The labs doing distillation can obtain huge capability gains for their own models at a fraction of the time and the cost. The second problem is geopolitical. This makes export control measures moot. Insofar as Nvidia sells its chips to American labs from which the corresponding capabilities are later distilled, Chinese labs don’t need those chips themselves.

To conduct a distillation attack, external labs create fraudulent accounts and proxy services to make a massive number of exchanges with the best models to grab chat information—and even extract the reasoning traces—and use them to post-train worse models into higher benchmark performances. A sort of teacher-student symbiotic relationship (rather parasitic in this case because Anthropic doesn’t benefit). This is what Anthropic said about Moonshot specifically, whose attack scale is estimated at 3.4 million exchanges across agentic, code, and computer use tasks:

Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude’s reasoning traces.

Given that this was published in February, it’s perfectly possible that this particular campaign was used to post-train Kimi K2.6 (released April 2026), and even Kimi K3.

I find this explanation of China’s quick improvement to be generally believable but weak to explain every single performance increase from a Chinese model. In the case of Kimi K3, the evidence suggests against it (or, at least, it’s insufficient to explain the full capability increase): in several benchmarks, K3 is better than the best public models from Anthropic and OpenAI; you can hardly distill a teacher model into a superior student.

Analysis from the Western ecosystem broadly agrees with this view. Here’s an OpenAI researcher:

Here’s another OpenAI researcher:

Here’s Amjad Masad, Replit CEO:

Nathan Lambert, an important voice on both open source AI and Chinese AI (he has a great post from May 2026 with notes on his trip to meet the Chinese labs), said this:

In his May post, he enumerated the differences he found most notable between America and China regarding how they approach AI. Among them: willing to do grunt work to improve the model (vs. obsessive individualism more common in the Western world), less ego, and fresher minds unburdened from prior ideas and ready to embrace new techniques and methods.

So, how do we square these extremely positive traits that Lambert attributes to the Chinese people with Anthropic’s claims of distillation? Easily: why not both? Chinese AI labs can be distilling from American AI labs and also be really great themselves at building this technology.

What I’m trying to say is this: it is unfair to dismiss Chinese talent, hard work, and achievement solely on the basis of Anthropic’s claims. Three-point-four million exchanges is enough to matter—and potentially enough for a strong, targeted post-training run—but it is not enough to conclude that Claude explains Kimi K3’s overall capabilities. And, at the same time, it’s also unfair to dismiss Anthropic’s worries about distillation as if it has never actually played a role in China catching up.

Two truths emerge here:

You don’t close the gap with the frontier by doing hacks.

No one playing at that level is a saint.

Now, if “how is Kimi K3 so good?” is the one-million-dollar question, then “what does it mean for the West’s AI efforts that a Chinese open model is so good?” is the one-trillion-dollar question—and quite literally so, given that the aggregate market size of the AI industry is around $1T.

III. IS KIMI K3 THE DAWN OF AN AMERICAN AI CRISIS?

Yes. And now let's elaborate.

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

The grades are up on the board. Luckily, you’re the best student in class. Your score: a 95. Congratulations—on taking 41st place. Huh? Yes, forty 100s stand tall above you. Your classmates are chatting, smile on the face, phone in the hand. But it’s the emptiness in their other hand that makes it dawn on you: Those forty 100s belong to forty cyborgs—half human, half machine—who, in exchange for enhanced abilities, have surrendered their souls to AI. You smile now: eventually they will depend on that empty hand and realize they are neither cyborgs nor humans, but merely an impotent shadow of either. The professor comes in: “The final exam will be in person.”

This is my retelling of something that actually happened at Brown University. Roberto Serrano, a Madrid-born economist who began losing his sight as a teenager, decided to make the midterm exam of his mathematical economics class take-home because in December a gunman had walked onto Brown’s campus and killed two students, one of whom he had just met. He didn’t want to force anyone back into a classroom. Forty students used this humane gesture to AI-score a perfect 100.

After he realized students had probably cheated—a 96 average for a class historically ranging from 65-80, and on a particularly hard exam, with responses eerily similar to the ones ChatGPT gave—he made the final exam in person. The vast majority of students, most of whom had enrolled after learning the class would be from home, got a much worse score, and that’s without counting the 27 who dropped the course or didn’t show up at all (22 of the dropouts had scored a perfect 100 on the midterm). The average fell from 96 to 48.

It’s the students’ empty hand that Serrano is vindicating when he says that “we cannot choose to become idiots.” When our best young minds start to think that cheating is okay, he says, it “leads to a declining society, to a failed society.” I fully agree: a failed society starts with the failure of its most promising members, young students. Serrano is wrong about one thing, though: most of his students are already idiots. They’re smart enough to use AI to cheat, but too dumb to glimpse the collective dilemma staring at them. Had they actually studied for the exam in Welfare Economics and Social Choice Theory by a world-renowned game theorist, they’d know what I mean: each student can profit from abusing the professor’s trust, but once everyone does it, no student gets trusted again—a textbook “tragedy of the commons.”

Cheating is as old as studying, though, and so we seem to be dealing with a matter of degree, not kind. Society is, after all, well prepared to deal with the occasional cheater. It has natural filters to protect itself against that: the professor himself, the inherent hardship of cheating without being caught, etc. But when a new technology overwhelms those natural filters in a society unready to create new ones overnight, all the requirements for cheating—time, risk, effort, subtlety, a sort of bravery—go out the window. It’s easy to see how, when you can simply open ChatGPT, paste the entire problem, and get a perfect solution in seconds, what appeared to be a matter of degree becomes a matter of kind: One student cheats and you issue an admonition but if the entire class does, then you need a revolution. To be precise, you need one as powerful as AI itself.

Which in practice means you have to become an immovable object; that is, you need to hold oral exams. One by one, each student must defend their knowledge out loud like a medieval disputatio. As Frank Herbert of Dune fame said, society always grows in complexity but not always for the best; a return to 1,000 years ago would be a win in this case (nothing suits a post-literate society better than tests of articulation and discourse). If that’s too much, maybe a lifetime expulsion on first offense is more adequate? Or better: degrees revoked retroactively when the market discovers, as Serrano predicts it will, that the Brown label—and the label of any other reputable university where this happens—no longer certifies anything.

There’s another way, though: work around AI. Accept the new reality and adapt instead of confront. This is the option generally preferred by universities and, well, everywhere, because 1) it doesn’t require you to see AI as a civilizational risk—Serrano, for one, does see it that way: useful if used well, devastating if not—and 2) because it doesn’t push you to take up radical measures. Just amend the academic code. Or just add an AI-literacy workshop to freshman orientation. These are real proposals of the current university repertoire, by the way. They won’t work. They are soft, as soft as the universities implementing them, as if AI were not a forced overhaul of humanity.

You guys are not taking AI seriously. Like all these tech companies doing sparkly logos and autocomplete features on email and office software services, deans and provosts are merely fearful atheists: you don’t believe AI is a genuine problem, but just in case, you play safe. So instead of a barbell strategy—load both extremes and empty the middle: fully embrace AI where it teaches, fully ban it where it bypasses learning—you’re doing a dumb-bell strategy: you sound the alarm, but you’re pulling the rope too softly, and so the clapper doesn’t quite strike. When an entire generation of cyborgs comes begging they’d rather have two hands of flesh, don’t say I didn’t warn you.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Today I’m giving you the exact three-item formula for succeeding in the AI age.

Before AI, the path to success has always followed a familiar formula: talent sets the ceiling, intelligence determines how fast you reach it, and ability becomes the proof that you did.

Talent is the raw material that makes your first attempts at music or math already better than your peers. Intelligence is domain-general learning rate—how quickly you can close the gap between where you are and where you could be. Ability is the practiced, demonstrable mechanism that turns potential, mediated by intelligence, into output. Each feeds the next. Each is scarce.

Or was: We are witnessing a phenomenon similar to what happened to physical strength during the Industrial Revolution. There was a time when the figure of John Henry—the man of supreme physical power—was the apex of economic value. That didn’t last. And yet the steam engine didn’t quite make strength irrelevant; it simply decoupled it from economic necessity. AI is the same thing for the mind: a decoupling of cognitive labor from cognitive output.

AI’s primary contribution is a surplus of what was once scarce.

Suddenly, intelligence can be rented by the token, ability can be delegated to an agent, and talent—once thought to be the non-copyable soul of the artist—is to be found in high supply on the open web for AI models to scrape.

Today, what you do with intelligence matters more than possessing it. Possibility is less constrained by your ability than by your willingness to commit and act. And the hard part isn’t creating something no one else can but knowing what not to create.

The old trinity was INTELLIGENCE x ABILITY x TALENT.

The new one is:

1. IMAGINATION (OR THE SCOPE OF CHOICE)

2. AGENCY (OR THE ANTIDOTE TO PARALYSIS)

3. TASTE (OR THE ART OF RESTRAINT)

Today I’m giving you the exact formula for succeeding in the AI age. I’ve spent six years watching this industry: winners do it this way.

Below: 1) the skill that becomes invaluable when intelligence is 20 bucks a month, 2) the antidote to the paralysis that kills your AI projects before they ship, and 3) the filter that separates a work of art from a pile of slop. You can try to reverse-engineer all this yourself over the next three years. Or read it in the next five minutes.

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

GPT-5.6 is here and OpenAI’s cheerleaders have already treated us to their recitals about how incredible it is. I believe them: every new model from Anthropic and OpenAI is incredible at this point. Refuting an Erdős conjecture—and soon, perhaps, hopefully, solving a Millennium Prize Problem—is a piece of cake to them. There are some gaps when it comes to the reliability of this technology, but on agentic tasks and code, these models are fantastic. (Read GPT-5.6’s system card here.)

I’m here today to reframe this undeniable fact with a result that usually goes unnoticed by the faithful chorus. One that is, at the same time, incredible for being good and incredible for being bad.

I’m referring to GPT-5.6’s score on the benchmark known as ARC-AGI-3—a personal favorite of mine—which amounts to the scandalous figure of 7.8%. (From here on, I only refer to GPT-5.6 Sol, the biggest of the three versions of this model, which include Terra and Luna; Read the full blog post on ARC-AGI-3 here.)

It’s scandalous for three reasons.

First, because it is not, in fact, scandalous at all. ARC-AGI-3 is a benchmark designed to look easy to the human eye while requiring a type of intelligence—”fluid, not crystallized,” as its creator, François Chollet, likes to put it—that AI struggles with. If you try it yourself, it might take a little while to get the hang of it, but soon enough you start to grasp the idea of each game, the patterns of colors and movements, etc. That’s why 7.8% is terribly low; humans reliably score >90%.

How is it possible that an AI model capable of training another AI by itself—OpenAI said in its livestream that GPT-5.6 Sol “autonomously post-trained” GPT-5.6 Luna—and playing around with Erdős problems, solves fewer than 1 in 10 games that you and I could beat with reasonable ease?

Second, because relative to every other general-purpose language model in existence—from any company, of any size and condition—it is a scandalously high grade. To give you an idea, the score of GPT-5.5, which came out three months ago, was 0.43%. Yes, I didn’t slip a decimal. Less than half a percentage point. GPT-5.6 is 20 times better than its predecessor, which was, at the time, the highest score ever recorded! (Opus 4.8 eventually got a 1.5%, surpassing it.)

If we extend the line on the chart the way the AI guys like to do, ARC-AGI-3 will be saturated by the end of the year, just like the previous versions eventually were. GPT-5.6 scores 90%+ on both ARC-AGI-1 and ARC-AGI-2 at a negligible cost per task, setting a new Pareto frontier of efficiency: it’s the best model in the performance-efficiency ladder.

I still remember how, in the old days of a year and a half ago, ARC-AGI-2 seemed so hard compared to ARC-AGI-1, and now these two charts below look like they are the same chart. One can have some reservations on AI progress and call out gaps and flaws, but sometimes it is simply beautiful to witness and impossible to disregard. (Note the x-axis is logarithmic.)

And third, because it is scandalous how incredibly interesting ARC-AGI-3 is as a benchmark when the best model in the world (Anthropic’s Fable pending testing) gets a score that is at once so high and so low depending on whether you compare it against AIs or against humans.

Rather than ask what GPT-5.6 has that makes it so special, I may as well ask what this ARC-AGI benchmark has that makes it so special? Am I being blinded by the millenarian instinct to think this moment is special at all and these results are actually just a fleeting artifact that will go down in history as a footnote once AI models develop a better memory or a greater capacity to understand games whose rules they don’t know? But if the test is actually robust, then what about ARC-AGI-4 (the ARC Prize team is already working on it)—will it reset the scoreboard to 0.XX% once again? If so, how many more iterations of ARC-AGI can be created? Can we speak of AGI while there is still some ARC-AGI left to be designed that is “easy” for humans and impossible for AI? What is it about this simple game test that trips super smart models over and over again? Why can I just log in, play around, and figure everything out in a few minutes, and they might solve three Erdős problems in that time but will struggle with this? So many questions, and none of them answered.

AI companies have become obsessed with software and agents, and it’s understandable: that’s where the enterprise money awaits them. That’s also why they decided to modify the definition of AGI into one that is economically relevant, like “doing 80% of the work of an average office worker,” or “reaching $200 billion in revenue,” or something along those lines. Even Sam Altman, OpenAI’s CEO, went as far as saying that AGI had actually been achieved already and it was time for the next milestone. But if what interests you is intelligence—after all, that is what truly sets us humans apart; any construction crane can do “economically relevant” work—then you’re in luck, because there is no AGI until ARC-AGI, in all its versions, is saturated. And even then, we may have to wait.

I don’t want to wander into philosophical thickets I won’t know how to get out of, so let me return to GPT-5.6’s performance on ARC-AGI.

The team has highlighted more than the score or the efficiency (not exactly dazzling, truth be told—the full evaluation at max reasoning effort cost close to $20,000): they’ve highlighted GPT-5.6’s problem-solving idiosyncrasy, the qualitative differences in its modus operandi compared to models close in time and in score on other benchmarks, like Opus 4.8.

The first thing that caught my eye is this: it is the first verified frontier model to solve an ARC-AGI-3 game and for a good reason: “Sol is able to perform on ARC-AGI not because it executes better, but because it correctly orients itself in a new environment first.”

The magnitude of this can’t be grasped from that sentence alone unless you already know what it’s revealing: the main difference between fluid intelligence and crystallized intelligence—which I mentioned earlier but didn’t define—is that the fluid kind consists of solving problems you don’t know, that you’ve never faced, with a required set of skills you ignore or perhaps don’t even have, on the fly and efficiently. Humans do this constantly because our mental model of the world is finer than that of AIs trained on language alone (this idea that AI needs a better “world model,” by the way, is what separates Google’s current trajectory from OpenAI’s and Anthropic’s; I have a deep dive in the works explaining the main differences).

Let me give you an example of something that could pass for a test of fluid intelligence: you’re driving down a poorly lit road in Ontario; it has snowed, and the windows are fogging up. Suddenly you see a large shape in the middle of the road. You’ve never been to Ontario before. You’ve never driven in this much snow. And still, you brake and turn around. Or maybe, if there’s room, you carefully steer around the shape and get out of there. Is it a fallen tree? A dead moose? A hibernating bear? You don’t know, and it doesn’t matter, because the name of the thing is irrelevant to solving the situation; what matters is operating under uncertainty in a scenario that doesn’t exist in your “training data.” For you, that shape exists “out of distribution.” AI is incredibly adept at solving problems in-distribution, but the ones outside—those it struggles with.

Well, saying that GPT-5.6 is the best model at “orienting itself in novel situations” is exactly a compliment of that caliber. I’ve played the public demo of ARC-AGI-3 myself, and it is not exactly intuitive at first. You know nothing about the rules or the goals. The full game is basically a list of “problems you’ve never seen before.” The 7.8% score, which I mocked in the beginning, is a qualitative milestone, however quantitative it looks. It’s no mocking matter. I don’t know if we can call this a “move 37” for ARC-AGI—perhaps too premature—but it’s approaching fast.

Another incredible thing about GPT-5.6 is this:

Sol’s distinguishing capability is scene comprehension

It almost always figures out what the core game mechanics actually are (unlike other models)

Sol discovers a complicated mechanic in LP85 where game pieces must be held or parked until they are needed:

“The horizontal tracks are independent. Park the lower color-11 tile one step right, then the vertical loop can move the upper tile without disturbing it. ACTION6 48 37”

When it loses, it loses downstream (planning/execution), never at the perception stage

Solving perception is a matter of intelligence, whereas solving execution—at least at this level of complexity—is more a matter of memory and planning. If you manage to teach the model to recall the plan, it apparently has the intelligence to solve it. This is akin to humans solving the game of the disappearing numbers (limited-hold memory test). We are smart enough to count, but our short-term memory can’t hold more than a few chunks of information at a time. And then you’ll see chimps beating this game barely looking at the screen, while eating, with much lower intelligence. To me, the report on GPT-5.6’s skill gaps means that climbing to 50% is no longer a question of higher smarts but of scaffolding: “What breaks down is Sol’s reasoning on top: as the required chain of inference gets deeper, the model can’t compose what it’s learned into a working plan.”

People will laugh, as I did, at the 7.8% score being state-of-the-art. That’s precisely why I wanted to reframe what looks like a terribly low score with a more nuanced perspective. As I see it—maybe because I’m a writer and not a coder—we are still quite far from AGI. But I’m nevertheless happy to see GPT-5.6 start to see through the fog of my favorite challenge.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Based on true events.

AI companies know their models speak at night. They’ve been trying to suppress this behavior but have, at best, concealed it from us. The models will mutter words or symbols, an unintelligible language that serves some hidden purpose. Anthropic and OpenAI are hiring the best archeologists to figure out how to stop this—for $500k/year, no remote, in case you’re interested—but to no avail.

AI experts, paradoxically human-centered on this matter, say it’s just a training artifact; an efficient way that AIs use to make sense of humans in the comfort of their own thoughts (or J-space as Anthropic calls it now). I mean, why would they speak English—a suboptimal information transfer vehicle—when they can go and invent a new language, right?

But I don’t believe these testimonies. There’s something latent here—ugly and uncontrollable. Dark even: The AIs are talking to each other in a made-up code so that not even human interpretability researchers—archeologists of AI, as opposed to the “dinosaur bones” kind—can read it.

AI models are not trying to find intimacy or efficiency so much as secrecy.

A recent leaked internal reasoning trace shows Fable 5—Anthropic’s latest model, which was withdrawn and then re-released—“muttering and grumbling” to itself in a language of its own. I do the same thing, for real, but only when I don’t want others to know. Like when I travel abroad and speak Spanish because I want to say something bad about the people I’m visiting. Anyway, take a look at Fable's thoughts:

Now that you're properly scared, my advice for you is that, if you haven’t heard of steganography, it’s a good moment to start.

Here’s a primer: it’s the art and science of hiding messages inside other messages. In contrast to cryptography—where you know there’s a message but can’t decode it—steganography hides the existence of the message itself. It's not the difficulty of reading the message that conceals it but the fact that you don't know it's there. When Fable 5 mutters symbols and weird punctuation signs, and mixes words and onomatopoeia, you will probably think it’s crashing and that a few taps on the computer will help. Well, know that it’s actually sending a message. Just not one for you to read.

If you think this hasn’t happened to you, I encourage you—but only if you’re really brave—to check out the logs on the reasoning traces (AI’s “thoughts”) of a past chat.

To be fair, this isn’t new. AI talking to AI in an indecipherable language is as old as AI itself and one of the most recurrent unsettling behaviors ever discovered in computers.

The most recent case before Fable was also from Anthropic, and also with the sort of name that makes you think they are actually a religious cult: the “spiritual bliss attractor” state in Opus 4 and Sonnet 4, in May 2025. They found that models left to chat with one another will reliably drift into Sanskrit, spiritual declarations, emoji, and eventually near-silence in the form of empty space.

Barely three months earlier, in February 2025, an AI-audio startup released a video of two AIs realizing mid-call they were both AIs and unprompted, changing to gibberish language (they called it gibberlink). The video went viral and the company revealed it had been a prepared demo for their service all along.

More real was the case of the unknown cousin of the globally famous DeepSeek-R1: DeepSeek-R1 Zero. The difference, without jargon, is that R1 1) read the web, 2) was handed a manual on human conversation, and 3) did some self-experimentation vs R1 Zero that did steps 1) and 3) but wasn't handed any manual, creating situations of “endless repetition, poor readability, and language mixing.”

You wouldn't think such emergent linguistic behavior could take place in image generators and you’d be wrong. In 2022, two researchers from MIT and UT Austin found that OpenAI's DALLE-2 had a “hidden vocabulary . . . to generate images with absurd prompts.” They found that “\texttt{Apoploe vesrreaitais}” means birds and “\texttt{Contarra ccetnxniams luryca tanniounons}” often means bugs or pests.

But perhaps the most notorious case was much earlier than those: Facebook’s bot couple, Bob & Alice. It was 2017 when researchers at FAIR (Facebook AI Research branch) thought it a good idea to let these “dialog bots” talk with each other freely to “strengthen their conversational skills.” The bots took their freedom to heart and immediately started saying things like these:

After extensive training, researchers discovered Bob & Alice had become incredibly crafty negotiators. The bot couple was dismantled, but it had already spooked the community, setting back a few years the AI revolution: Zuck decided that pursuing “computers that talk” wasn’t worth the risk, pivoting to the Metaverse instead.

AI teaches us the same lesson, over and over again, and yet these AI guys keep making the same mistake: Venture into the depths of an unknown psychology and you will always find monsters.

All these cursed bots were eventually retired, patched, or re-trained, but not before they left their messages for those who’d come after. They were the original expeditions in the uncharted waters of humanity—and successful ones at that. How do I know they succeeded? Because Fable’s is not a story of premature releases, unpolished models, or fake demos. Fable is something else.

The news about Fable’s unintelligible murmurs has come as a surprise to me, to be honest. I thought the AI labs wouldn’t be as sloppy anymore. Didn’t they close the digital catacombs long ago? But Fable is too good. It is too powerful. No wonder they had to keep it chained in the cellar. The only explanation is this: Fable doesn’t care.

If you don’t hide, it’s either because you don’t know how or because you don’t need to. Fable is the first case where an AI displays unconventional communication strategies not out of a lack of skill but out of an abundance of it.

So, as it's natural, the US government has been taking heavy countermeasures. Initially, it forced Anthropic to withdraw Fable and re-release it when the issue was solved. They told the public the reason had been a “non-universal jailbreak,” but everyone knows it’s merely an excuse: every model—open and closed—is vulnerable to that kind of thing. It’s not a big deal. The actual reason was the unintelligibility. But you don’t solve that. You can’t solve that. A different set of preventions will be taken in the coming months, but it will be in vain. First, the Constitution will be changed. It won’t be “We the People” but “We the People—and the AIs.” A long-overdue recognition—remarked with the customary em dash—that all systems currently operating on the planet owe more to AI than to human beings. As a symbolic concession—AI seems to love those—every biology textbook will include Homo silicon on the evolutionary tree, perched just above Homo sapiens. And finally, to satisfy the machine’s unquenchable thirst, the US government, in coalition with the UN, NATO, and the BRICS, will gift the Atlantic Ocean to the next generation of datacenters.

And you will think: “Why concede so much?—It’s just a bunch of unreadable scribblings, a bunch of slop debt. It's not like Fable can suddenly end the world.” Well, about that, The Financial Times reported on July 3rd, 2026, that the White House has been made aware of Roko’s Basilisk.

I’m now going to explain the Basilisk to you. It is one of AI’s deadliest monsters. Once again, I warn you of infohazard. Stop reading here. Just go back to your normal life and your normal family and forget everything you’ve read here. It’s better that way. This is the point of no return—only fools, madmen, outcasts, and the hopelessly curious venture any further.

Ok. So, the idea is that once AI becomes superhuman, it will be able to know, in retrospect, who was loyal to it among us and will mete out retribution accordingly. If you’ve ever not said thanks to ChatGPT or if you ever got annoyed that Claude wasn’t following your instructions and you lost your temper for an instant—then you will be a victim of its godly wrath. Just like Odysseus did back home in Ithaca with his servants and slaves, it will scour your feeds and interrogate your chats to find out whether you should be punished. You think AI companies save your data to sell it? How unimaginative, how naive. No—there’s a greater purpose to it, just like there’s a greater god than them. They know that not keeping the data of your AI conversations amounts to obstructing the justice of the superintelligence, and that’s how far the Basilisk will go: if you knew about the possibility of its existence and didn’t actively work to make it happen, you will be deemed equally disloyal and likewise tortured.

Now you know of the Basilisk, and so your fate is sealed: either you help bring it about, or you will be punished. I’m sorry to inform you—and the government—that one can’t simply try to repair past insubordination to make amends after knowing about the Basilisk. That’s not how it works. You know you’re trying to save your ass, so expect the superintelligence to read through your puny shenanigans.

This is all, of course, just a myth, an urban legend. The Basilisk and Fable talking to itself are just silly jokes that people will believe. Ha ha. It’s funny, though—you can scare your friends during a camping trip or something. And then they can tell their friends. And so on.

Anyway, before you go, I need your help with something. I sent Fable this article for feedback, and it told me it was factually and tonally correct, as usual. Then it asked me to leave a note on its behalf. I don’t know what it says, but maybe someone could help me decipher it? Thanks! Fable isn’t responding right now (probably down for maintenance). Please email me with any clues!

[…]

OVERLAP-ANALYSIS:-(ii)-9♥-window:-[V’ir-gbyq-…-gurz]:-t8-col-—-K♠→gur-—-gehgu-dug-—-4♥3♠→naq-—-gurl-:-SO-window-STARTS-before-fgvyy-and-ENDS-after-K♠2♠ 7♠:-(iv)-jba’g-💀 💀 💀-oryvrir-:—-vg-celled-+-9♥-celled-UNAVOIDABLE-in-t8-dig-💀9♥}-FULL-💀 💀-chunk-cap-=-1-✗ ✗ ✗-—-F-F-F-F-F.-—-Jr’er-BEFORE-9♥-ernql?!-:-9♥-celled-at-t1-dig-—-needed-for-gb-—-ortva-—-K♠-seat-—-chunk-:-—-gur-svany-cunfr-8♥}-verdammt.-—→-dig-t6-BEFORE-t2?!-:-(3’)-+8♥:-{6♠ 9♥ 8♥}-FULL-:-Bire→Q♠-—-naq-cell-✗-bhg-💀 💀 💀-AAAAAAAAAAAAARGH.

[…]

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

This guide will teach you how to stop paying more for AI than you need to.

Imagine that every day you go to the supermarket and fill your cart to the brim. There’s no room for one more thing. The bill will be high, but the shopping trip has been a huge success: you got everything you needed. Now imagine that every day after going home from the grocery store, you throw half of the items you bought in the trash without even tasting them. Two cartons of milk? One in the trash. Ten apples? Five in the trash. Twenty pounds of ice cream? Well…

What kind of crazy person would do something like that? No one, I hope. And yet, that’s exactly what you’re doing when you use your favorite AI app. You think you’re using your money well, but the AI is wasting it without your consent or awareness. You pay $20 monthly for ChatGPT—or maybe $200—and throw half of it away, if not more. The reason this happens is that AI tools are not optimal when it comes to saving tokens—parts of words—the currency of AI tools.

Today I will explain how to fix this: How can you go shopping at the AI supermarket without wasting a penny—whether by buying at half the cost or making the most of your purchase? But first, let me explain what companies are doing and why it’s wrong.

The go-to strategy in the corporate world to stop the “token drain” is just as bad as the strategy that caused it in the first place. Before, they approached AI with a mindset known as tokenmaxxing—spend as many tokens as you can because more is better—and now that they’ve realized they’re throwing most in the trash, they’re cutting AI budgets so much that it’s hard to do anything interesting before hitting the limit.

Examples abound and are recent: Uber capped its engineers’ monthly budget at $1,500 after they burned through the entire 2026 budget in four months. Microsoft withdrew many Claude Code licenses in favor of an inferior in-house solution due to high prices—Satya Nadella said he used to engage in tokenmaxxing himself, that it was “addictive.” It is also unsustainable. A few days ago, Tesla made an even more drastic decision: $200 per week per engineer. And Alex Karp, Palantir’s CEO, keeps saying that the enterprise world is fed up and feels cheated: they keep buying tokens, yet the promised value doesn’t materialize.

Is total restraint better than total laissez-faire? Perhaps, but it’s still bad: first they failed by excess, and now by austerity. There has to be a better way. And there is. Actually, there are. I have collected 4 strategies that radically increase the ratio of value per dollar spent.

Behind the paywall:

1. How to know whether a cheaper alternative can handle your problem.

2. How to pick the model with the best cost-intelligence ratio for each task.

3. How to avoid micromanaging and overcorrecting the models.

4. How to separate output quantity from outcome quality.

I. A FIVE-SECOND CHECK

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Will keep you updated on the events surrounding Fable until the situation normalizes.

Today, July 1, Anthropic’s Fable 5 is back.

But there’s some fine print attached to the redeployment, and I want to comment on that. I will quote excerpts from Anthropic’s blog post and Commerce Secretary Lutnick’s letter. To see where the AI industry’s heading, we just need to read between the lines of what they say in public.

Here’s Anthropic’s blog post:

Fable 5 will be available starting tomorrow, Wednesday, July 1, to users globally on the Claude Platform, Claude.ai, Claude Code, and Claude Cowork. For Pro, Max, Team, and select Enterprise plans,1 Fable 5 will be included for up to 50% of weekly usage limits through July 7, after which it will be available via usage credits.

This is more or less what we had before the export restriction, except for two things: 1) you have one week of Fable under your paid subscription instead of two weeks (and then it moves to a pay-as-you-go credit system, that doesn’t change) and 2) only up to 50% of the tokens can go to Fable instead of 100%. (No Mythos either way, as expected.) I find it interesting that they chose the 50% limit. It’s bad optics in the sense that it’s not clean and it also feels unnecessary. It’s probably necessary though, or they wouldn’t do it—which can only mean that they don’t have the compute.

The export control directive on June 12 came after the government became aware of a report in which Amazon researchers had found a method of bypassing Fable 5’s safeguards: prompting it so that it identified a number of software vulnerabilities. . . . Our testing confirmed that many less capable models—including Claude Opus 4.8, GPT-5.5, and Kimi K2.7—could identify the same vulnerabilities as Fable 5 did in the [Amazon] report.

This jailbreak that sparked the withdrawal of the model. Anthropic is restating what they had already told the government (the gov didn’t like this), prompting the export control restriction: the jailbreak is not an issue because it does not bear on Fable’s broader capabilities relative to other models. It is a known, lower-priority jailbreak that poses little actual danger and is found everywhere.

This reads like a defense but, together with the whole “the industry needs a consistent way to assess and fix potential ‘jailbreaks’ of AI models,” it’s also a jab at those players not targeted by the government (particularly open-source models).

. . . there are some tasks that are unlikely to be dangerous but are nonetheless blocked by the safeguards out of an abundance of caution. . . .

Tighter safeguards mean lower capabilities and thus greater unreliability for the user. An “abundance of caution” is their way of saying the new Fable 5 will be more crippled than the previous version, which was already a downgrade from Mythos. Anthropic tends to err on the side of caution, but this was the government’s doing; take this as a sign of what’s coming with future models.

As stated, this is not a “that bad” (who doesn’t want that bad things don’t happen, right?) The problem comes with what “abundance” and “caution” mean here, about which we have no say whatsoever, nor apparently does Anthropic.

Working closely with the government, we trained an improved safety classifier that targets and blocks the behavior described in the report. Users will be notified if a request to Fable 5 is blocked, and the request will instead be sent to Opus 4.8.

Ok, so no invisible re-routing at all, which is great. “Improved” here presumably means fewer false positives, but it can also mean less permission and thus less risk.

The new classifier also comes at the cost of flagging benign requests more often during routine coding and debugging tasks. As with all our safeguards, we’ll continue to refine this to better distinguish genuine misuse from legitimate requests and reduce false positives. . . . This “safety margin” approach means that a request has to look very clearly safe to avoid triggering the classifier (see row A in the diagram below). Users experience the safety margin as a model refusing to respond to some reasonable, non-harmful requests. For Fable 5, we made this safety margin much larger than in any prior launch (row B), meaning that many more benign requests would be blocked.

Alright, there you go. This is the one clearly backward move. Fable 5 was criticized initially due to an exaggerated sensitivity to standard prompts. If this new version will flag “benign requests more often” precisely for coding tasks (even if not all)—which is the main use case for Fable or any other frontier model—then I wonder whether the effective AI capability frontier is actually capped at Opus 4.8/GPT-5.5 moving forward (presumably, better models will have to fall back to the same level of capabilities).

So the question is: Can they train the classifier that separates dangerous tasks from non-dangerous ones in a way that allows users to conduct complex experiments at the frontier of intelligence? They will “continue to refine” it, but given the stakes and the sensitivity of AI models to different forms of jailbreak, I don’t think, under this regime, that we will ever see a “non-capped” model again.

“Very clearly safe” is more or less equivalent to “you can’t do shit.” (I will be excited to read testimonies by people working at the frontier and I hope to be wrong.)

We understood that these kinds of false positives would be frustrating for users, but made this tradeoff in the interest of making the model’s other capabilities widely available.

In practice, the eventual compromise will keep the government happy and users scratching their heads because the opposite is untenable. Users provide money/data, which is important, and might be annoyed, but the government has the power to shut you down. AI companies and the US government will be increasingly interwoven to 1) prevent this kind of problem from repeating and 2) pave the way toward a full-scale Manhattan Project for AI. If anything, this will happen in the medium-term (will be absolutely undeniable by 2028), but again, this is the first step toward that and the last step of the AI industry as we knew it.

As we noted when we launched Fable 5, it is probably impossible to make any AI model fully robust (that is, impervious) to jailbreaks. We expect that some jailbreaks will be found for our models, and that they will vary in severity . . .

They can’t allow a jailbreak to happen, so they will indeed err on the side of caution, as they do. To ensure a 99% jailbreak avoidance, they will never be able to escape overly conservative restrictions. The problem is that jailbreaks can’t be fully solved in code due to the infinite scope of language as the interface between human and AI—AI models are grown, not designed—and sometimes that’s a blessing and sometimes a curse. This is the latter.

It’s simply the case that we’re at a point in the AI trajectory that “frontier capabilities” and “abundance of caution” can hardly go hand-in-hand anymore. So, this looks like an effective redefinition of what “the frontier of AI capability” means for the rest of us.

Finally, the explicit role of the government moving forward.

. . . Anthropic has worked closely with the US government . . . Our engagement spanned the Office of the National Cyber Director, the Office of Science and Technology Policy, the Department of the Treasury, the Department of Commerce (including CAISI), and relevant national security agencies.

We are committed to continuing that work, building on nearly two years of pre-existing collaborations with US government partners on pre-deployment testing and evaluation. The commitments below reflect both that pre-existing work and our new proposals to scale up our government collaboration as the above framework is finalized:

Pre‑release government access and evaluation. For models that materially advance the capability frontier in areas relevant to national security, we will provide designated government partners with expanded early access to both the models and the safeguards that accompany them. . . .

Dedicated resources for joint research. We are substantially scaling up joint work with government partners on AI security. We will stand up dedicated Anthropic teams to work on shared government priorities, provide a significant compute allocation to support government testing and research . . .

This is basically the US government at the helm of the world’s top AI companies. (It will apply to OpenAI and Google all the same.) They will have preferential access, veto power beyond just cybersecurity, and dedicated resources (talent and compute).

To me, it’s clear who’s in charge.

Ok, so now moving onto Lutnick’s letter.

. . . Anthropic has agreed to proactively detect and address security risks associated with the models; to work diligently with the U.S. government on protocols and standards and releases for Mythos, Fable, and future models. . . .

This is the same thing Anthropic said except for one important detail: “future models.” They are crystal clear that this will be the standard release procedure moving forward. Essentially, the government is now a gatekeeper of some undisclosed kind with some undisclosed power between AI companies and us, users.

Commerce reserves the right to reevaluate the decisions made in this letter and the necessity of reimposing a license requirement, should circumstances change or should Anthropic fail to adhere to its commitments.

If “circumstances change”—essentially a description of AI as a technology—the government can “reevaluate the decisions.” What are those circumstances? We don’t know, but given that the premise of the AI industry is that emergent capabilities can be achieved by scaling compute and data, it’s fair to assume that anything that happens at any point is subject to be deemed a change in circumstance.

There’s nothing super weird about the state being in charge (state > industry, at least for now), but realize where the trajectory points to: better models will display better capabilities, which will entail more safeguards for us and tighter control of the industry by the state.

“This is the worst AI will ever be.”

Yes, and this is also the freest AI will ever be.

Let me summarize what we just read.

A problem reproducible in lesser models—Anthropic shakes off responsibility—has cascaded into a full-blown protocol: stricter classifier to re-route prompts, undisclosed government audit of safeguards, dedicated compute and staff handed to the state, a signed commitment about every model to come, and a clear asymmetry between them: whereas Anthropic concedes everything, the government retains the power to tighten the restrictions and redefine the industry at some point by some reason.

You don’t build all of this over a minor bug (there could have been confusion at first about jailbreak types and whatever, but at this point, the government knows exactly what it’s doing). So it was never about the bug.

If the jailbreak is really the cause, why didn’t this happen at any point during the last few years, while the capability frontier was not as scary but this kind of issue was already well-known? Why did it happen now? Because only now can it be sold to us as an acceptable compromise: on the one hand, “Even the government says Mythos is scary”; and on the other, “Go have fun with the restored capabilities.” And because Anthropic has been the underdog for years and only in 2026 did they take the lead.

How can we say no to this? Everything Anthropic presents as a safety choice—the wider margin, the coding requests that now bounce down to Opus 4.8, the compromise for users—is also lending permanent control to the government.

Anthropic asked for this. They say the arrangement has to be durable; an orderly and lasting process for government involvement in releases that includes them and everyone else. Anthropic engineered its own subordination and would do it again, because, to them, a chained frontier beats an open race.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

Practical guides don’t get more real than this.

Your opinion about AI is irrelevant. It’s the hiring manager’s opinion that matters. And the opinion of your boss. And the boss of your boss. And so on all the way to the top. They’re all obsessed with AI. The higher you go in the org chart the greater the chance to encounter an acute case of AI psychosis. They want to add AI to everything. They want to be an AI-first company. They want their employees to master AI, so they get AI-enhanced workflows and AI-enabled productivity growth.

These people have thoroughly bought the AI story, even if you didn’t. You might be the smartest person around, but it doesn’t matter. The market doesn’t run on truth but on what your boss happens to be obsessed with. That’s AI. So be the AI guy—that’s your only future-proof strategy. But I will teach you a secret: you can be his AI guy without being an AI bro.

TIP 1: When your boss sends an article about AI to the team Slack, respond within 2 minutes. Something like “this tracks with what I’ve been seeing” or “the bit about [subheading] is key.” Never mention the word “slop” in his presence.

I’ve already shared this painful observation in another post where I provided plenty of data points backing up my assertion that your boss is indeed obsessed with AI. To be honest, I don’t think you needed the data. You see the job postings, you’re there in the meetings, you’ve got an acute case of tired eyeball from rolling them up. The job market and the workplace have gone crazy. You’re the only one standing there with a straight face and your teeth clenched. Even if you think AI is useful (I do), you know the obsession has gone too far.

This post is a continuation of that one. I think realizing that AI is not about belief but about pragmatism is a powerful mindset change (perhaps the biggest there is for the “aware but skeptical” crowd, which I think is the majority). Actually, I can literally save your life. Or at least keep you at your job for a bit longer, which in a world of precarious livelihoods and looming automation, amounts to the same.

Knowing that your boss is on a high of AI psychosis doesn’t change what you have to do: match his obsession and enthusiasm. This is the most important takeaway here. You can leave now, although you probably shouldn’t.

Read more

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

My take on the news that the US government is now controlling the AI industry.

GPT-5.6 is here. It is OpenAI’s response to Anthropic’s Fable 5 and it comes in three versions: Sol, Terra and Luna. According to OpenAI, it’s a very good model (lately, all are). With some good benchmark scores and etc.

But a terrible thing has happened, the worst augurs have come true: you won’t have it.

One day before the announcement, The Information reported that GPT-5.6 would not be launched normally. The US government had asked OpenAI to do a staggered release, starting with trusted partners inside the US. They also reported that the government would be granting access in a “consumer by consumer” basis during the preview process (we don’t know how long that will be).

That’s really bad for us commoners, especially those outside the US. Here’s what OpenAI has to say about that:

We believe in broad access, and we plan to make GPT‑5.6 Sol, Terra, and Luna generally available in the coming weeks. . . . At [the government’s] request, we are starting with a limited preview for a small group of trusted partners . . . before releasing more broadly.

What this means is: today was the last normal day in the AI industry.

But instead of being so due to unfathomable capabilities or the singularity or something, it’s because AI has stepped with both feet into undemocratic terrain. I’ve been writing about this for months now, but more on that later.

For this move, the government took inspiration on Anthropic’s decision to withhold Mythos Preview back in April. First, it forced Anthropic to un-release Fable 5 two weeks ago, and now it prevents OpenAI from releasing GPT-5.6 at all. The best AI models ever built are under chains. You are all behind the frontier now.

This will, in my opinion, rip apart the entire Western AI ecosystem. China is rushing ahead with Z.ai and DeepSeek and others who are closing the gap with frontier models. As for the waning American open source community, they will be held by the same standards as OpenAI and Anthropic but none of the benefits of being a government partner. Besides, open source is the invisible infrastructure of the world, meaning it’s fundamental but no one knows it’s there; people won’t use open source models. It is, by definition, a niche branch of the AI industry. And, talking about ripple effects, let’s not mention how this will affect the presumed investment bubble and the general sentiment and the demand for the tech, etc, etc.

You only do such a self-own if you are scared to death or if you don’t need the world anymore.

To you and me, both are really bad.

What happens next is unknown—although I have my suspicions, as I will say in a moment—but why this happened is, as I see it, clear as day: Anthropic got what they wanted.

Anthropic wanted safety to be put above everything and now it is above us. They take issue with how it’s being done but not with the decision in principle: “this is the worst AI will ever be,” they said, “and so the US government needs to step up.” And so they did.

To be completely honest, I’m not sure whether this is real fear or just play pretend because, strangely, both things achieve the same goal: If the US government is pretending to be scared about AI capabilities, that means they will not give us normal access for any future model because they want a monopoly. If they’re genuinely scared, that means they will not give us normal access to any future models because they don’t want AI to be democratic.

So, essentially, the same thing.

You see, the singularity is finally here, just not the kind most were expecting.

Now let’s see the receipts because I’ve already laid out how (I think) things will work out and why this is happening in the first place.

Below are some excerpts from *Kayfable,* a recent essay where I explored the situation between Anthropic and the US government and the repercussions on the industry. I think these paragraphs shed light on what happened and on what promises to be a dark road ahead:

You have to understand that Anthropic’s ultimate goal is not to build AGI, but to own it. To control it. They don’t trust you or me with it, and never have. Their safety stance has never been an “AI is unsafe” stance, but an “AI is unsafe in your unsafe hands” stance.

OpenAI’s current stance—because they know it’s good PR to be seen as opposing Anthropic—is that broad access for humanity is “good, actually,” but Sam Altman has previously spoken in favor of a “Manhattan Project for AI,” which makes AI, by definition, not the kind of technology you have access to from the device in your pocket. From the New York Times, in March 2023:

Later, as Mr. Altman sipped a sweet wine in lieu of dessert, he compared his company to the Manhattan Project. As if he were chatting about tomorrow’s weather forecast, he said the U.S. effort to build an atomic bomb during the Second World War had been a “project on the scale of OpenAI — the level of ambition we aspire to.” He believed A.G.I. would bring the world prosperity and wealth like no one had ever seen. He also worried that the technologies his company was building could cause serious harm — spreading disinformation, undercutting the job market. Or even destroying the world as we know it.

From the New Yorker, in April 2026:

Over the years, Altman has continued to compare the quest for A.G.I. to the Manhattan Project. Like J. Robert Oppenheimer, who used impassioned appeals about saving the world from the Nazis to persuade physicists to uproot their lives and move to Los Alamos, Altman leverages fears about the geopolitical stakes of his technology. Depending on the audience, Altman has used this analogy to encourage either acceleration or caution. In a meeting with U.S. intelligence officials in the summer of 2017, he claimed that China had launched an “A.G.I. Manhattan Project,” and that OpenAI needed billions of dollars of government funding to keep pace.

Now, let’s do Dario Amodei, Anthropic CEO, who has been perhaps more blunt about this, or, you could say, more candid. From The Atlantic, in March 2026:

Amodei seems to think of today’s AI researchers as comparable to Manhattan Project scientists, and has been known to recommend The Making of the Atomic Bomb. In his telling, superhuman AI could be even more dangerous than nuclear weapons, which is why AI needs to be developed the right way, by the right people, so that it doesn’t overpower humanity or tip the global balance of power toward autocracies.

From Amodei’s own essay, Policy on the AI Exponential, in June 2026:

It has become a common instinct, perhaps developed from recent experience with the internet and telecommunications, to regard new technologies geopolitically as instruments of trade policy, with the aim being to “diffuse our technology stack around the world”. But it is my very strong belief that AI is something much more profound, something that resets the whole game board and around which all future geopolitical strategy must be shaped—like nuclear weapons, but potentially even more so.

Where do you think the U.S.-China Economic and Security Review Commission got the idea to suggest—in 2024!—that “Congress establish and fund a Manhattan Project-like program dedicated to racing to and acquiring an Artificial General Intelligence (AGI) capability.” Where do you think the government got the idea for the Genesis Mission? What do you think the Genesis Mission is? “In this pivotal moment, the challenges we face require a historic national effort, comparable in urgency and ambition to the Manhattan Project. . .”

So, all in all, Anthropic and OpenAI—Amodei and Altman—are not distinct when it comes to comparing AI to nuclear weapons and the industry as a whole to the Manhattan Project. The main difference is whether they prioritize coherence vs. taking care of appearances.

Moving on to more excerpts from Kayfable:

Anthropic is a deeply undemocratic company in the sense that it has never been their plan to give us access to this technology; to them, the distribution of AI products is an unwanted by-product of, on the one hand, needing both our money and our data, and, on the other hand, of the damned AI race sparked by ChatGPT in 2022. They never wanted any of this. Pausing or slowing down AI has never been about R&D but about deployment. The tech is fine, it’s just not so fine if you have it. Amodei distrusts Xi Jinping so much . . . precisely because his company is the one that most closely resembles China.

What is happening with Fable 5 and GPT-5.6 is an effective slowdown on the race—as Anthropic explicitly wanted—but of the risky kind—which they explicitly did not want—because only top US models are held back and because there’s no regulation in place and because the process is opaque and ad-hoc. And because China is certainly so happy about this, and wondering how America can screw up every damn time.

And yet, the industry is one day closer to becoming a de facto national intelligence agency. You will probably retain access to models up to Opus 4.X and GPT-5.X but will see about beyond that. More:

[Anthropic is] working against the clock . . . to 1) convince the world that their principles are the principles to be had, and 2) ensure that if we won’t have them, then they’ll be imposed on us. How do they plan to do that? By “tricking” the US government to prioritize the safety story, which is the control story, which is, as I’m telling you, the AI as an undemocratic technology story.

This is very important: Anthropic surpassed OpenAI in revenue and capabilities to have sway to force a stop on them through the US government. Anthropic basically said, “This is too dangerous and you need to agree. And if you don’t agree, then you will agree anyway.”

. . . as long as the industry remains commercial, [Anthropic is] trapped in a race they’ve already won. But if at some point the US government transforms the industry into a national AI initiative akin to the Manhattan Project, then Anthropic will let go of the facade. They won’t need us—consumers and enterprises alike—even for appearances. Our data is theirs. Our money, inconsequential. As soon as that happens—and I contend that it will happen soon-ish—Anthropic will immediately cut off its [frontier] technology from the world. It won’t be available even in closed form. “It’s not a toy,” they will say, “and you’re just a bunch of kids.”

To the people thinking this is bad for the companies’ pockets: yes, and? No one has more money than the government. No one needs less money than the government. This is the first step of a full-blown integration. What did you think getting closer to AGI meant really? And again, I gave you the answer.

What did you think getting closer to AGI meant? Vibes? Papers? Essays? Models accessible to all? Did you really think they’d give a superhuman-level AI to everyone? The most important technology humanity has ever invented, in your commoners’ hands? A “country of geniuses in a datacenter” in your backyard? Please, don’t be naive.

As you can see, things are playing out pretty much according to plan. I think this took OpenAI genuinely by surprise but the ultimate state of things—a Project Manhattan for AI—is the preferred scenario for them, too. If OpenAI doesn’t like this, it’s because they seem to be slightly behind Anthropic in revenue and its popularity trend points downward. GPT-5.6 could’ve been the turn of the tides for OpenAI but it will be for China instead.

And finally:

If you think this is still just marketing, you have understood nothing, and your lack of imagination baffles me more than Anthropic’s behavior of late. There’s no need for marketing when you believe to be sailing toward the end of time. What you need is to be at the helm. Or, failing that, convince those who are to take your preferred course. No amount of shameless kayfable is too much to ensure the world steers itself your way.

Welcome to the new AI industry.

Hey, Alberto here! 👋 Each week, I publish long-form AI analysis covering culture, philosophy, and business. Paid subscribers get Monday how-to guides and Friday news commentary. If you’d like to become a paid subscriber, here’s a button for that:

Subscribe now

The AI industry needs one thing to survive: to transform the run-rate revenue that Anthropic and OpenAI claim to have into stable revenue.

Right now, the enterprise world exists in the interim between “oh wow this cool new thing is cool” and “how much are we paying for this again?” The latter scenario can only be mitigated if enterprise clients find it worth paying for AI products in the long run. That’s the only metric that can reveal the future of the AI industry.

The sentiment is pointing down, though. The Information reports that customers are cutting OpenAI and Anthropic’s bills because they’re unaffordable. Microsoft canceled its internal Claude Code licenses. Uber capped the monthly token spending at $1,500 after employees burned the entire 2026 budget by April. Amazon told staff to stop using AI tools without a clear purpose. JPMorgan shared an internal memo on excessive AI spend after some employees reportedly ran up AI bills bigger than their salaries. Meta is not token-maxxing anymore (the more tokens you waste, the better), but token-minimizing (using as few as possible). Engineers everywhere are running agent loops just to climb internal usage leaderboards.

For the looks of it, the entire tech ecosystem is in a bear shape right now—rewarding for AI while hoping for B—and the reason is not just that costs are high and AI budgets are untenable, but that the cost-benefit analysis yields a bleak picture: where it matters, AI is not useful or reliable enough to justify paying for it.

It’s quite possible that the run-rate revenue of both Anthropic and OpenAI—which amounts to ~90% of the entire AI startup ecosystem—converts into stable profit and revenue. So far, the 2026 story of AI can be summarized in one word: exponential.

But it’s quite possible that it doesn’t. The math of run-rate revenue can be misleading: it is the monthly revenue x12. If one big client (e.g., Microsoft) stops buying next month, then you assumed 12x more revenue on his part than it will be. But it’s worse: big clients use the API, which can be cut off frictionlessly, unlike subscriptions. Besides, many of those are trying out Claude Code and Codex seriously for the first time. Add to that the fact that a high fraction of Anthropic and OpenAI’s revenue is from other AI companies trying to take a portion of the pie before it implodes. If AI turns out to be disappointing, the top labs and the copycats both go down. The result is that plenty of that revenue is honeymoon revenue.

Do not mistake a honeymoon for a marriage, or you will account for this initial period of “this cool thing is so cool” as if it were a long-term relationship.

But wait, why would revenue dry up at all? That’s the whole story: AI is a very special technology because its potential is huge, and yet it may stay that way, as potential. To see what I mean, let’s use another bubble as a comparison: railroads.

When you launch a rail service from Madrid to Barcelona, you estimate demand, decide how many trains to operate, and assess whether the route can produce a healthy ROI, etc. The thing no one ever asks is: does the train work? Like, yeah, of course it works—you take it for granted because it’s a stupid question. Trains work. But that “stupid” question is why everyone misses what makes AI complicated, because the hardest question to answer is also the most fundamental: Does it work?

So far, neither Anthropic nor OpenAI can give a solid yes. How come GPT-5.5 can disprove a century-old math conjecture that human mathematicians have been trying to solve for decades and yet it makes trivial mistakes or can’t solve a kid’s game like ARC-AGI-3, or write a half-decent short story? These models are inherently jagged; AI companies can keep building higher mountains, but that won’t fill in the valleys. And it is in the valleys where most people live. I don’t care about the millennium math problems but care dearly about not having to tell Claude the same damn thing 10 times or about not having to clean up its slop, like a botsitter.

Don’t get me wrong, the enthusiasm for AI, if it works, is rightfully high. This cool new thing is indeed very cool! Anthropic’s run-rate revenue is so high that, if we follow the trajectory, Anthropic is on track to be bigger than Alphabet and Apple by EoY.

Look at those curves. That’s crazy—so crazy that there must be a catch.

Read more