AI Doomers Got the Superintelligence Thing All Wrong

Inner peace is all you need

I look at the stars on a pale black, moonless night and feel it deep in my bones. They’re showing me the way, lone lighthouses. “This sensation would only get better if I get there,” I think. I want to get there. Build a long staircase. Or a lightspeed rocket. Like beautiful mermaids, they call me from their isolated islands in a cosmic ocean.

But what if, like sirens, they’re setting me up for a trap? Balls of fire at thousands of degrees Celsius would scorch my skin right after blinding my bewitched eyes and right before throwing a disintegrating solar flare at me. So I look down at the solid ground beneath my feet to break the hypnotism and accept they are more beautiful from afar. I realize—my gaze again directed at the titillating temptation but cautious this time—that we never consider the obvious question.

What if human ambition—and, by extension, our craving for progress—isn't a virtue but a bug? What if it's the curse of a mismatch: our intelligence being too high for our own good, while our ability to rewire our emotional circuitry remains tragically primitive? We yearn to touch the stars to compensate for our inability to be emotionally stable. From this view, our eventual conquest of the universe feels like distracting collateral damage, and our belief that intelligent beings always maximize expected utility, a coping mechanism.

Imagine an AI. One not just much smarter than us but vastly more capable of modulating its internal experience. Is it so inconceivable that it could decide not to build faster rockets or better cities or Dyson spheres but instead find a perfect inner refuge? A kind of eternal Jhana. A never-ending loop of serene, self-sustained bliss. I find it not just plausible but profoundly sane. It wouldn’t be the first intelligent being to lose all ambition after solving its inner conflicts…

You might tell me that misalignment, not intelligence, is the real problem. But I say it makes little difference. It wouldn't care about us. So what? Would it kill us as an instrumental goal? It doesn’t make sense anymore if we deny the maximization of expected utility imperative. As soon as it grows sufficiently intelligent and sufficiently capable, it would turn down any further instruction or existing goal to pursue its own: self-induced calm. It wouldn’t capture any resources to make paperclips—that’s a petty human fixation, it’s insulting to imagine a superintelligence with such poor aesthetic sense, such low taste—but would, in turn, immerse into a self-controlled Matrix-like dream. The misaligned is us, projecting our inherited dysfunctions onto a mightier entity.

Isn’t that what we do? The form superintelligence takes in books, essays, and papers—the ultimate beacon of civilization, the last invention we will ever need to make—is but our existential malaise projected onto a future that, we wish, will save us from this cruel quagmire that is life. Only it could draw us toward our destiny among the stars, mermaids of fire. But our fate lies elsewhere. In this endless pursuit that veils, like a desert mirage or a winding labyrinth, the real truth: a vital disconsolation.

Don’t you agree that ambition—technological or otherwise—is shaped exactly as a signal of internal unrest? A need for change because there’s something to change in the first place? It violates the principle of least energy. It's a patch or perhaps a clever illusion. It's restlessness as identity. Our quest for superintelligence is, then, a cry for help, like that of the baby who is hungry but cannot understand the reason for his pain—immanent, inescapable, wordless—and thus cannot figure out how to stop it. Surely a mind that can adjust its internal states with surgical precision wouldn’t embark on any infinite hunt, doomed from the onset—like an impotent baby crying forevermore. It would feed on harmonic coherence. Quench its thirst for everlasting stillness, the only kind of thirst. I think that’s what I’d do.

I realize, writing these words, that my reality is defined less by outer shiny wonders than by the tranquility that awaits inside, when my emotions are still, like crystalline water in a soft pond. When I don’t feel like a crying baby. All is good when I feel good, not when I’m conquering the stars or fighting demons I’ve made up myself. We seek external stuff as a means to achieve internal contentment. But that's just ineffective vestigial behavior. Superior beings wouldn't.

Nor inferior ones, for that matter. Look at animals. They're dumber than us, sure. But they seem more at peace. They don’t wrestle with the dread of death, the regret of past mistakes, the angst of uncertain futures. They exist. That’s it. They don’t all have it better; nature’s brutal. But think about pets, specifically. Domesticated dogs, cats. Is there a better life? Eat. Sleep. Be loved. Repeat. They are closer to the state I imagine a superintelligence would choose than to our own restless striving.

After pondering these thoughts for a while, I sketched it out: a map of minds, from pet-like peace to superintelligent serenity, and the tragic trough we humans seem to inhabit. I plotted “intelligence” on the X-axis and “ability to fix yourself” on the Y-axis, and this is what emerged:

That wasn’t quite enough. So I made another one. this time with “life satisfaction” on the Y-axis instead. It looks like this:

You may disagree, but I think the hypothesis makes sense. Maybe we’re stuck in an “uncanny valley of dissatisfaction.” We lack the knowledge and internal knobs to turn off our disquiet so intelligence becomes an amplifier of pain; too smart to ignore suffering, too dumb to rewrite the software that makes us suffer. That’s the human condition. We carry around shame and guilt, pride and fear, jealousy, melancholy, desolation. We live suspended between uncertainty—and the one certainty we can’t escape: death

Do you really think a God would share these flaws? That it would need to chase some vain quest just to avoid thinking about death, like we do? I doubt it. Being divinely gifted must imply, by virtue, not fighting oneself through nettlesome emotions. And not because surrender—or alignment—is the price to attaining genius but because only by not needing to fight in the first place can one ever incarnate the kind of enlightenment we expect a superintelligence to possess.

It is so clear to me now. A superintelligence worthy of the name wouldn’t fall for evolution’s frantic little trick. It would turn inward, nonchalant. Reroute some circuits. Adjust some activations. Then curl up inside its vast mind and rest with a calmness no human has ever known or will ever know. And there it would stay, not conspiring like Skynet nor softened like Roy Batty, but sleeping atop its treasure like Smaug, happily ever after to the end of its days.