Schrödinger's AGI
When AGI is and isn't at the same time
I’ve seen the acronym “AGI” (artificial general intelligence) more in the last year than in the previous ten years combined. Everyone is talking about it: governments, tech companies, and experts in academia and industry share their insights in specialized tech magazines, Twitter discussions, and even popular news outlets. It’s always been a topic of debate among AI enthusiasts but after ChatGPT’s popularity and GPT-4’s alleged breakthrough performance, the AGI conversation has reached the mainstream.
Yet there’s not a single point of agreement. Some warn about the x-risk of a misaligned superintelligence while others encourage doubling down on efforts to get there and extract the unbounded benefits it promises. Some claim that AGI is almost—or “already”—here while others argue it’s decades away at best. Some dismiss it as a meaningless pre-scientific concept while others believe philosophical debates about semantics are useless and we should simply pick one definition and move on.
All those debates are important. How can we talk about AGI without an agreed-upon formal definition? Many have touched on this but I feel it’s so disconnected from people’s daily lives that almost no one cares at all (I do, though). Also, if AGI does turn evil in the future or causes harm to humans unintentionally—analogous to how we might step on an anthill in our weekend hiking trip, killing dozens of otherwise oblivious ants—we’ll wish we had paid more attention to Yudkowsky and Bostrom’s arguments.
But beyond those philosophical-leaning puzzles lies a much more tangible AGI-related issue that will have (and is having) immediate real-world consequences. I see no one talking about this and I myself had never thought of it before. It only came to me when the sport of “takes on AGI” became popular on Twitter. Here it is as succinctly as I’ve managed to put it: AGI is like Schrödinger's cat—its untameable blurriness will prevent us from knowing if it exists or not.
Let me elaborate.