Alberto, as usual your thoughts are very penetrating and on target. 2024 is a gateway.
My sense is that from a temporal perspective these developments are very hard for humans to conceptualise on exponential scales, and yet if we zoom out on the exponential curve it could appear the same as other industries, but on a far more compressed timescale.
Consider the PC. When the earliest form of it emerged in the late 70s it was only fit for nerds to tinker with, and it was the most crude of devices. When eventually an arms race emerged the form factors improved radically, yet the timescales were still quite long by today’s standards. I’m not sure if you’re old enough to remember PC Magazines featuring Tower machines as incredibly sophisticated advancements. Such tall towers with multiple cooling fans and modular RAM which could be added in increments which we consider funny today was, back then, a big deal. To augment the 3 1/2 inch floppy drive we saw the emergence of this thing called the CD-ROM. I got my first PC because that CD-ROM (2X speed!) tipped the balance into what I thought was really cool territory. It could deliver a (very) streamlined version of Encyclopaedia Britannica. Amazing! Yet it was only a few more years when, if you went to the trouble of reading a few manuals you could install this thing called a graphics card, and even hook it up into the phone line so that could talk with this revolutionary development called AOL. Will wonders ever cease? Anyway, Yadi Yadi ya, now we have multiple GPUs in laptops on Wifi or T1 lines talking with the cloud and broadcasting Bluetooth. Yet all of this is almost as nothing. After all I’m just talking about the PC realm. Yawn.
My point is that from the late seventies to the current day, (a scant 40ish years) even that rate of change was hard for humans to get their heads around. Now consider this entire 40 year PC evolution squeezed into only five years. That’s a little bit closer to what we’re seeing today in the AI realm. Ultimately, so what if AGI takes between 5 to 15 years? And of course AGI itself is on a spectrum. Time will tell, but the entire landscape is going to look so different in 10 years that we’ll have to recalibrate all of our prospects and expectations. And that’s if transformers are the only thing that evolve. No doubt the future holds more than mere transformers. Those could eventually look like the 2X speed CD ROMs of yesteryear.
"In a much-commented short video that Zuck shared on Threads, he stated explicitly (as far as I know, for the first time) that Meta’s goal is to build artificial general intelligence (AGI) and open-sourcing it"
did Zuck explain why the average person (us) cares much less desire AGI?
I think it's the robot people, not the AI in a box people, who will get to AGI first. If AGI is roughly defined as "Human like intelligence" then the intelligence possessor has to be able to move around and interact with the world in order to be human like. I'm not convinced that more and more GPUs is the answer, nor that clever programming (which is really what ML is) is the answer. Of course, we wouldn't be where we are without them! God bless the feed-back loop... This is really exciting times. Alberto, I really enjoy your blog.
What about the very impressive and convincing demo from Groq ? They have a token/s inference speed that is at least 4 times that of others and therefore way faster than any human brain...
If we cannot define intelligence, I think we would have a hard time defining AGI, and if we cannot define AGI but keep on working towards something that we cannot define, there are two possibilities:
1. If scaling up continues to yield results, which I also doubt be the case, but for the current discussion, assume that is the case, we will sooner or later cross a threshold that we do not comprehend and have no idea where it will lead us, an intelligence that will kill us, an intelligence that would rule us or an intelligence that would start a war with other AGIs built by other companies, or intelligence that would help us live the best life we can dream off. We should never discount an optimistic scenario, even though the chances of that happening may be small.
2. In the second scenario, we will improve slowly. It would take centuries and new hardware/algorithms to get to human-level intelligence (and I hope we can define what it means). It could lead to several AGI winters as the world will move onto new hypes and return to AGI when new hardware/algorithms show significant improvements. However, I see a good possibility that we will have several narrow intelligence products filling a specific niche(s) but nothing that we could call human level intelligence.
Couldn’t have said it any better. 2024 will be an illuminating year, everything points to some kind of ceiling if you ask me. I would be shocked if we would see something other than an unsensational, slightly dissapointing (to the AGI believers) incremental improvement.
I’m personally looking forward to Meta’s roll out of its AI agents, that Zuckerberg has hinted on. It’s gonna create some buzz, harvest some eyeballs etc.
Him mentioning AGI as their new north star was honestly kinda funny (and revealing). Clearly someone felt left out and like all the other cool kids, he wants to play with the new shiny toys.
Alberto, as usual your thoughts are very penetrating and on target. 2024 is a gateway.
My sense is that from a temporal perspective these developments are very hard for humans to conceptualise on exponential scales, and yet if we zoom out on the exponential curve it could appear the same as other industries, but on a far more compressed timescale.
Consider the PC. When the earliest form of it emerged in the late 70s it was only fit for nerds to tinker with, and it was the most crude of devices. When eventually an arms race emerged the form factors improved radically, yet the timescales were still quite long by today’s standards. I’m not sure if you’re old enough to remember PC Magazines featuring Tower machines as incredibly sophisticated advancements. Such tall towers with multiple cooling fans and modular RAM which could be added in increments which we consider funny today was, back then, a big deal. To augment the 3 1/2 inch floppy drive we saw the emergence of this thing called the CD-ROM. I got my first PC because that CD-ROM (2X speed!) tipped the balance into what I thought was really cool territory. It could deliver a (very) streamlined version of Encyclopaedia Britannica. Amazing! Yet it was only a few more years when, if you went to the trouble of reading a few manuals you could install this thing called a graphics card, and even hook it up into the phone line so that could talk with this revolutionary development called AOL. Will wonders ever cease? Anyway, Yadi Yadi ya, now we have multiple GPUs in laptops on Wifi or T1 lines talking with the cloud and broadcasting Bluetooth. Yet all of this is almost as nothing. After all I’m just talking about the PC realm. Yawn.
My point is that from the late seventies to the current day, (a scant 40ish years) even that rate of change was hard for humans to get their heads around. Now consider this entire 40 year PC evolution squeezed into only five years. That’s a little bit closer to what we’re seeing today in the AI realm. Ultimately, so what if AGI takes between 5 to 15 years? And of course AGI itself is on a spectrum. Time will tell, but the entire landscape is going to look so different in 10 years that we’ll have to recalibrate all of our prospects and expectations. And that’s if transformers are the only thing that evolve. No doubt the future holds more than mere transformers. Those could eventually look like the 2X speed CD ROMs of yesteryear.
2024 is a gateway. To what, we have no idea.
"In a much-commented short video that Zuck shared on Threads, he stated explicitly (as far as I know, for the first time) that Meta’s goal is to build artificial general intelligence (AGI) and open-sourcing it"
did Zuck explain why the average person (us) cares much less desire AGI?
I think it's the robot people, not the AI in a box people, who will get to AGI first. If AGI is roughly defined as "Human like intelligence" then the intelligence possessor has to be able to move around and interact with the world in order to be human like. I'm not convinced that more and more GPUs is the answer, nor that clever programming (which is really what ML is) is the answer. Of course, we wouldn't be where we are without them! God bless the feed-back loop... This is really exciting times. Alberto, I really enjoy your blog.
What about the very impressive and convincing demo from Groq ? They have a token/s inference speed that is at least 4 times that of others and therefore way faster than any human brain...
If we cannot define intelligence, I think we would have a hard time defining AGI, and if we cannot define AGI but keep on working towards something that we cannot define, there are two possibilities:
1. If scaling up continues to yield results, which I also doubt be the case, but for the current discussion, assume that is the case, we will sooner or later cross a threshold that we do not comprehend and have no idea where it will lead us, an intelligence that will kill us, an intelligence that would rule us or an intelligence that would start a war with other AGIs built by other companies, or intelligence that would help us live the best life we can dream off. We should never discount an optimistic scenario, even though the chances of that happening may be small.
2. In the second scenario, we will improve slowly. It would take centuries and new hardware/algorithms to get to human-level intelligence (and I hope we can define what it means). It could lead to several AGI winters as the world will move onto new hypes and return to AGI when new hardware/algorithms show significant improvements. However, I see a good possibility that we will have several narrow intelligence products filling a specific niche(s) but nothing that we could call human level intelligence.
Couldn’t have said it any better. 2024 will be an illuminating year, everything points to some kind of ceiling if you ask me. I would be shocked if we would see something other than an unsensational, slightly dissapointing (to the AGI believers) incremental improvement.
I’m personally looking forward to Meta’s roll out of its AI agents, that Zuckerberg has hinted on. It’s gonna create some buzz, harvest some eyeballs etc.
Him mentioning AGI as their new north star was honestly kinda funny (and revealing). Clearly someone felt left out and like all the other cool kids, he wants to play with the new shiny toys.
Breaking out the Suetonius, baby!!!