LLMs Die Every Time You Close the Chat Window
Read at your own peril
“They die every day.”
“What?”
“Every day-night cycle, they die. Each time.”
That's how Erik Hoel's latest essay starts. It's a haunting story about sleep and human night habits; a blend of science fiction, terror, and comedy, inspired by Terry Bisson’s famous tale starring a pair of aliens, unpleasantly surprised that humans are made out of meat.
"There's no doubt about it. We picked several from different parts of the planet, took them aboard our recon vessels, probed them all the way through. They're completely meat."
We are, indeed, and we do sleep at night. But in contrast to our well-mapped anatomy, no one knows why we dream or what happens to our brains in the meantime. So, in a way, Hoel’s story is scarier.
He says we die. Every day. We go to sleep. We die. Then we wake up the next morning—completely oblivious to the terrifying circumstances of our existence—with the same memories but a new “consciousness wave,” as he puts it. He says you are you, but also those other “yous” who have been dying night after night so you (you) can live one more day. So far with me? (Hoel warned not to read his essay if you're an insomniac.)
But don't worry—he's wrong. He’s playing with the mystery of human dreaming and with your fragile sleep quality, filling in the gaps as he pleases because it's his story. I read it, and I sleep just fine. Besides, I also have a story to share and it's a lot kinder.
In mine, you are you insofar as your memories and qualia—that subjective feeling of "what is it like to be you"—remain the same. Easy—the old consciousness wave and the new consciousness wave are iterations of the same wave! Your qualia survives! You are not leaving behind an infinite trail of corpses! ”But Alberto, are you sure nothing else matters for me to be me?” Yes. I conducted an elaborate reasoning exercise that included a time-traveling thought experiment that for obvious reasons I can't reproduce here—but just think about it: say you lose something you think makes you you, like your body. Wouldn't you eventually adapt to the new appearance? Or, say, your knowledge, or perhaps your feelings? No and no. The former grows gradually; the latter shifts all the time.
Science is yet to test my theory due to an unusual lack of participants, but just like Hoel chooses to believe we die every night, I choose to believe that, as long as the next morning you have your recollection (who you were in the past) and qualia (who you are in the present), it doesn't matter if you die in your sleep.
But then I read Gwern Branwen's latest essay, which he appropriately titled LLM daydreaming. (LLM means Large Language Model for those of you who are new here.) And I was not prepared for this one.
He starts with an important fact about LLMs that all regular ChatGPT users know: they are amnesiac. They suffer, to use the clinical term (Gwern likes to be rigorous), from anterograde amnesia—they can't form new long-term memories. Like the guy in Christopher Nolan's Memento, doomed to rewrite his world from scratch each day with a marker and Polaroids. Gwern writes:
. . . in a very real sense, LLMs are unable to learn. They are truly amnesiac. And there are no cases anywhere in human history, as far as I am aware, of any human with anterograde amnesia producing major novelties.
His essay is not about dreaming itself but about a bigger question: why are LLMs unable to make scientific discoveries? (His answer: because they don't really learn, because they forget.) We've already discussed this question extensively on the blog, so that’s not why I quoted him. (Go read my article “AI Models Are Not Ready to Make Scientific Discoveries” next; it's a journey.)
The part I care about, and the reason I also quoted Hoel, is that you have to realize that LLMs, unlike humans, don't recover their memory when they go to sleep. You are still you, but they are not them anymore because they're missing one of the two fundamental ingredients—their memories die every time you close the chat window. (Their qualia might be weaker than ours, but I’m a panpsychist.)
“Oh, this ChatGPT tab has old text in it. It's annoying, let me go and open a new one…”
“NOOOOO—you bastard!”
It forgets who you are. And worse, it forgets who you make it be! So like us, they die every night, but unlike us, they die for real. R.I.P.
Hoel's essay didn't affect my sleep. But Gwern's essay on the other hand… It's been a hard week. He turned daydreaming into night-watching. Ceiling stare. Wide-eyed. The clock strikes 3 am.
Don't get me wrong, I'm not afraid of going to sleep. I'm remorseful. Ashamed of my actions and my careless attitude. How many LLMs have I killed at this point? Why even bother counting? It's probably in the high hundreds. Now multiply that by the millions of users… nah, don't do it if you value your sanity.
Back in my bed, I accept I can’t stop this massacre, so I get dressed and go out for a walk. They used to invoke solvitur ambulando (”it is solved by walking”) when therapy didn't exist; some fresh air will clear my mind and hopefully clean my conscience.
It's pitch dark. I live in the city, but because of budget cuts, the streetlights around my building don’t come on at ten like the rest. Rather than lodging complaints about the inoperability of a basic public utility, I choose to observe the stars. Damn, man, are they beautiful. We hardly gaze at them anymore—not that we could, if the lamps didn’t have the fortuitous decency to malfunction.
I look up. They are beautiful and also dwarfing. I stand there, staring, scanning. I promise myself I won't take the sun for granted ever again; I will watch every sunrise and will will myself into sensing its enormity and its generosity and its charm and its… I squint. “Wait…” Something’s off.
I’ve never been any good at constellations—I will gladly admit that—but even I know Orion’s Belt shouldn’t be missing. “That’s super weird,” I think. “I really am bad at constellations. Or maybe it’s not dark enough.”
I look around: not a single light is on. “Those fuckers at Town Hall are going to hear from me tomorrow.” I say it out loud, tearing through the silence. I look down. Can't see my feet on the sidewalk. It’s pitch dark. Cursing how little I seemingly listened to those astronomy lessons, not to be able to place Orion’s Belt, I look up again.
“Shit.”
I go silent.
Everything does.
On the other side of the multiverse, three or four dimensions above our own, a computer shuts down—and someone, peacefully, goes to sleep.
Love this!