I think buried in your analysis, or hidden behind the neutral terms of learning from human feedback is that these are explicitly economic engines/intelligences being built, not intelligences.
The goal (as you intimate) is to automate 95% of human work. Which means the spikes will be towards areas that “solve” the biggest game, which is wealth accumulation. All of the abstract models (chess, go, even chat) are being trained to solve this issue, so the “thing” we create will grow to that evolutionary attractor.
We are not training it to solve human problems like corruption, suffering or the movement of money, instead focusing on trading, automating jobs, and designing drugs. It is an inherent bias.
We could train for better problem solving with different data and different models, but we won’t.
Problem solving is intrinsic to economic value. You can't separate both. That's why LLMs are first problem solvers despite having provided little economic value to show for that skoll
I totally get it, but we have to include bias in any discussion of the shape of intelligence. At least for now, capital is an evolutionary pressure, even for general purpose problem solvers (by the way, general purpose is not equal to neutral)
While "spiky star" embodies intelligence's high dimensionality, we desperately need a metaphor for its lethal toxicity. What enables us to dominate the biosphere also turns our sleepwalk towards the existential abyss into a run. And, BTW, the closer we get the nicer the scenery.
I think we’ve not evolved the cultural vocabulary to articulate the difference between being “clever” and being an “intelligent” human. The Naval quote and the final few paragraphs start to nudge in that direction.
The challenge — and why I believe we may never want to have a good answer — is that once we define it, someone/something will build a program for emulating it. It will be like the final section of Asimov’s The Last Question as mankind succumbs to the Multivac/Cosmic AC and fully fuses its being to that of the machine.
“Cleverness" is a good but limited ability to learn a multitude of skills, whereas intelligence is unlimited in that it includes learning to learn (faster better cheaper).
The only part of us that does not want a good answer or "union with the machine" is the same part that is already completely an expression of the machine. The remainder is neither discussed nor discussable, but is immune.
I think buried in your analysis, or hidden behind the neutral terms of learning from human feedback is that these are explicitly economic engines/intelligences being built, not intelligences.
The goal (as you intimate) is to automate 95% of human work. Which means the spikes will be towards areas that “solve” the biggest game, which is wealth accumulation. All of the abstract models (chess, go, even chat) are being trained to solve this issue, so the “thing” we create will grow to that evolutionary attractor.
We are not training it to solve human problems like corruption, suffering or the movement of money, instead focusing on trading, automating jobs, and designing drugs. It is an inherent bias.
We could train for better problem solving with different data and different models, but we won’t.
Problem solving is intrinsic to economic value. You can't separate both. That's why LLMs are first problem solvers despite having provided little economic value to show for that skoll
I totally get it, but we have to include bias in any discussion of the shape of intelligence. At least for now, capital is an evolutionary pressure, even for general purpose problem solvers (by the way, general purpose is not equal to neutral)
While "spiky star" embodies intelligence's high dimensionality, we desperately need a metaphor for its lethal toxicity. What enables us to dominate the biosphere also turns our sleepwalk towards the existential abyss into a run. And, BTW, the closer we get the nicer the scenery.
“We’re all special stars” - Alberto Romero, 2025
I think we’ve not evolved the cultural vocabulary to articulate the difference between being “clever” and being an “intelligent” human. The Naval quote and the final few paragraphs start to nudge in that direction.
The challenge — and why I believe we may never want to have a good answer — is that once we define it, someone/something will build a program for emulating it. It will be like the final section of Asimov’s The Last Question as mankind succumbs to the Multivac/Cosmic AC and fully fuses its being to that of the machine.
“Cleverness" is a good but limited ability to learn a multitude of skills, whereas intelligence is unlimited in that it includes learning to learn (faster better cheaper).
I look forward to meeting this intelligence. Everyone I've ever known is merely clever.
Using Tao's disambiguation, I'd say all humans are intelligent
The only part of us that does not want a good answer or "union with the machine" is the same part that is already completely an expression of the machine. The remainder is neither discussed nor discussable, but is immune.