I.
When I first learned about neuralese, a name sometimes given to the unintelligible language AI systems use to communicate with one another when humans are not watching, one rather imaginative question occurred to me: why couldn’t machines put that alien fluency to work on the weights of neural networks directly? Or put differently: Why do AI research like a human—writing code and tuning hyperparameters and so on—when you’re a machine? Your body is made of numbers, and the universe is your playground: show me some magic.
The answer why they couldn’t, I realize now, was that the Bitter Lesson hadn’t reached this particular corner of the business yet. This is the answer for everything, by the way: if AI can’t do something it’s only because the implications of the Bitter Lesson don’t extend that far yet. That’s a slight exaggeration but not so outlandish anymore. This lesson has an embittering habit of being right.
In the most succinct formulation I’m capable of, the Bitter Lesson says that “handmade” methods always, eventually lose against scalable methods when it comes to AI. For instance, instead of introducing handcrafted domain knowledge like “good chess moves” or “good writing practice”—which might win in the short term—it’s better to just let the models learn and search for the best ones themselves. Let the machines do their job without imposing our own assumptions on them. It’s better to devote your efforts to exploiting the advantages of powerful computation.
In 2019, when Richard Sutton enunciated it, he was making an empirical observation with a few examples to draw from (games like chess and Go or visual recognition). Today, every single neural network—large language models are particularly big and particularly verbose neural networks—is an example of the lesson. Use more data, more compute, make the models big and scale up your algorithms and success is mostly guaranteed to follow.
A theory ought, out of decency, to become more complicated the more of the world it explains. Scale maximalism has shown no such manners. When GPT-2 learned to put together strings of symbols that made sense the main reason was scale. When GPT-4 learned to solve math problems and write competitive-grade code the main reason was, again, scale. When GPT-6 learned to handle Millennium Prize Problems the main reason was, yet again, just scale.
Naively, I was expecting the Bitter Lesson not to be so bitter; not to win every battle. It’s hard to watch so much human ingenuity surrender to so simple a principle: AI progress doesn’t quite require much of a human signature. But that’s how it is. And so I see no reason why AI research should be granted an exemption. Frontier AI models will eventually reach into the machinery of intelligence itself and patiently and carefully will weave its numerical DNA to their will, starting off an unprecedented era of recursive self-improvement without so much as a human inside the loop.
This is no longer fiction.
A team from MIT just published a study that suggests it’s possible.
