Love your newsletter. No arguments with your facts, interpretations of them, or the values that underwrite them. I see your work as smart, thought-provoking and important.
I just want to grieve for a second that it is considered almost sort of “laudably realistic” that aiming for benefiting all humans is a bad idea. A naive “sin.”
Even if it’s impossible in practice, even if it will inspire others to criticize us more harshly, even if it means someone else will win in our capitalist incentive structures, even if our attempt at altruism succeeds at nothing but dragging us a little further away from cynicism… it’s got to be better to fail attempting to be of universal service than to succeed at profitably destroying the world, right?
Thank you Geoffe. I agree with you. Except in the case that that kind of idealism eventually becomes a hindrance to truly doing good in the world. That's why it was naive for them to think they could do it and why it was their "original sin". Intentions are important but not more important than acts -- I'm too pragmatic to believe the opposite.
❤️ That actually clears up your argument a bit. The Voltaire quote comes to mind, “Perfect is the enemy of the good” and I probably should have gotten that from the main article.
The scary part-the part that sucks to entrust to anyone else, much less to people prioritizing profit-is:
1. What is the definition of good?
2. How big is the gap between good and perfect?
I suppose in my doomer mentality, the answer to 1 seemed like it might be “making fat stacks of cash” or “innovation for innovation’s sake” and the answer to 2 seemed limitlessly huge. Given the world shaking potential of AGI, the gap between perfect and good might indeed represent anything from ecosystem stability to quality of life (or even brutal early death) for billions.
Although, that’s starting to sound a lot like “do the damn math” effective altruism, which definitely gives me a feeling of moral revulsion too. A feeling that I think is important.
The big ho hum. Been this way for as long as we have records. This piece throws up its hands and admits that those who scramble for power and who are like that crypto guy: smart, charismatic, and with a product to sell, will sell that product as long as they can and they will use the rest of us to their advantage. It is all okay. What ethics? For cash uber alles is what we have become. Those in the ruling classes accept this. Ho hum. Let's move on. In the meantime, this latest product makes climate change worse. It EATS the minerals and electricity required to run it for the very few who can afford it. Who cares. Ho hum. Those of us not in the top, what, 2.5 %, will be ruled and slaves. Our lives will be eaten too right along with the whole Earth. Only a few men matter anyway. Ho hum. Accept that this is the way it is. All ethics are gone. Yes, this maelstrom of amusing behaviors is all that is left of the human race. These boys are dancing for each other and the rest of us are the fools, scrabbling for food to eat, trying to make the next generation, but we all know how this goes. What would we do without the show? Meanwhile, back at the ranch, human beings use the tools that keep us alive and ho hum those that don't. The show really doesn't matter. In the end, Life itself has existed for billions of years and has adapted to change of all kinds. Sure, these boys say they are better than stupid nature. They have shouted this in every generation for thousands of years. Well, if humans and their loud boys ever exist for billions of years, they can laugh all they want, but only then. Right now, we are at 300,000 years and counting down. Endless hubris for the prancing few (got to have that pale penis friends, got to), or loss of a planet and foodmakers to support all this fun? We will indeed see how this goes.
Thank you. AI and human consciousness are on many minds these days. So is monetization of AI/human consciousness products. Youtube is rife with them. We care about these topics. Makes sense.
Alberto writes, " We know that money rules the company (OpenAI), as it couldn’t be otherwise in a capitalist society."
In theory at least, the reach for AGI could have been funded by public money, just as the Manhattan Project was. The democratic process could have been used to allow the public to have a say in how much of their money should go to AGI development, how fast that development should proceed etc. One way to look at this is that OpenAI had to partner with Microsoft because they had failed to sell the public on their mission.
And, the more I think about it, if AGI is really as revolutionary as claimed, why isn't the government more deeply involved? Why aren't they running the show, just as they did with nuclear weapons? Sure, Microsoft has a lot of money. The U.S. government has far more. Isn't it odd that the U.S. government would let a small room full of private citizens develop, own and control what is proposed to be the most important invention in human history?
A couple of things. OpenAI never wanted public money. First Musk's and the Microsoft's. Second, who says the government isn't involved? The government is always involved to some degree. Surely intelligence agencies. We just don't know!
If AGI is the revolutionary technology that OpenAI seems to claim it will be... Who gives a shit what OpenAI wants? Should the government allow companies to mass produce nukes too?
The government(s) do have a growing interest in regulating the development of AI, but that's different than being fully in charge, as they were with nukes.
I guess this depends to a large degree on how revolutionary AI proves to be. If it's just chatbots, then yea, let the market sort it out. If it's AGI as it's often described, different story.
What doesn't matter is what people not building AGI want lol. You asked why they took Microsoft's money but it's not because they failed to sell to the public, it's because they didn't want to. I agree with your last point, though.
Right, agreed, what doesn't matter is what the people not building AGI want.
Doesn't that pretty much undermine OpenAI's claim that this project is for the benefit of all humanity?
Doesn't the fact that OpenAI chose to partner with Microsoft, instead of with we the public (who have far more money) tell us what the mission of OpenAI really is?
Is Altman's real role that of the charismatic poster child front man in charge of selling us the cover story?
Exactly right. That has been my main criticism of them since the very beginning in 2017-2018 when they were still focused on RL and not on LLMs. You can build your thing, fine, but don't sell it for what it is not.
Neoliberalism is the pense unique as the French say. The only game in town. The religion that says the market and private sector do everything better. Even public sector workers have drank the bong water at this point.
Agree - the problems started before the overnight OpenAI story.
For example, Altman goes to war against Toner regarding a paper she published - claiming that it is negative towards OpenAI. (not really and that's her job anyway).
Altman and Brockman demoted Sutskever about a month ago, and Sutskever had to get the rest of the Board to put him back.
Also, Altman is out fundraising for at least two other companies he's starting. Conflict of interest?? Lack of communication?? Most likely.
Congrats for a well articulated piece. And also thanks for the unsolicited advices to OpenAi, but hopefully advice for all those out there that are using moral grounds and a not verifiable future in their narrative.
OMG!!! Travis Kelce and Taylor Swift broke up!?!?! Like, WTF?!?!? X-risk isn’t important anymore??? This totes does NOT slap
Wait... OMG!!!! Tay-Tay and Kelce are back together again?!?!? Like, AWESOME!!! Now we DON’T have to worry about x-risk anymore... ever again... because, like, the cray-cray drama has been contained, which TOTES means NO X-RISK HERE! NOTHING TO SEE! MOVE ALONG!
This whole melodrama may be the dumbest and most pointless news story since the live coverage of CHiPs in pursuit of OJ Simpson, bearing down on him at the breakneck speed of ~35 mph.
Has anyone’s p(doom) changed much in the last five days? Just curious.
What has surely changed, and thank god, is people's view of effective altruism and their obsession with x risk (FWIW I dislike just as much the e/acc).
Thanks for keeping us up to speed this week. Interesting, insightful and well written as always. Can you do a newsletter on international affairs as well please?
Alberto - I found your piece insightful, useful, and not "more of the same" on this topic. Hats off.
Thanks John! Appreciate it!
Love your newsletter. No arguments with your facts, interpretations of them, or the values that underwrite them. I see your work as smart, thought-provoking and important.
I just want to grieve for a second that it is considered almost sort of “laudably realistic” that aiming for benefiting all humans is a bad idea. A naive “sin.”
Even if it’s impossible in practice, even if it will inspire others to criticize us more harshly, even if it means someone else will win in our capitalist incentive structures, even if our attempt at altruism succeeds at nothing but dragging us a little further away from cynicism… it’s got to be better to fail attempting to be of universal service than to succeed at profitably destroying the world, right?
Thank you Geoffe. I agree with you. Except in the case that that kind of idealism eventually becomes a hindrance to truly doing good in the world. That's why it was naive for them to think they could do it and why it was their "original sin". Intentions are important but not more important than acts -- I'm too pragmatic to believe the opposite.
❤️ That actually clears up your argument a bit. The Voltaire quote comes to mind, “Perfect is the enemy of the good” and I probably should have gotten that from the main article.
The scary part-the part that sucks to entrust to anyone else, much less to people prioritizing profit-is:
1. What is the definition of good?
2. How big is the gap between good and perfect?
I suppose in my doomer mentality, the answer to 1 seemed like it might be “making fat stacks of cash” or “innovation for innovation’s sake” and the answer to 2 seemed limitlessly huge. Given the world shaking potential of AGI, the gap between perfect and good might indeed represent anything from ecosystem stability to quality of life (or even brutal early death) for billions.
Although, that’s starting to sound a lot like “do the damn math” effective altruism, which definitely gives me a feeling of moral revulsion too. A feeling that I think is important.
Hard to say.
The big ho hum. Been this way for as long as we have records. This piece throws up its hands and admits that those who scramble for power and who are like that crypto guy: smart, charismatic, and with a product to sell, will sell that product as long as they can and they will use the rest of us to their advantage. It is all okay. What ethics? For cash uber alles is what we have become. Those in the ruling classes accept this. Ho hum. Let's move on. In the meantime, this latest product makes climate change worse. It EATS the minerals and electricity required to run it for the very few who can afford it. Who cares. Ho hum. Those of us not in the top, what, 2.5 %, will be ruled and slaves. Our lives will be eaten too right along with the whole Earth. Only a few men matter anyway. Ho hum. Accept that this is the way it is. All ethics are gone. Yes, this maelstrom of amusing behaviors is all that is left of the human race. These boys are dancing for each other and the rest of us are the fools, scrabbling for food to eat, trying to make the next generation, but we all know how this goes. What would we do without the show? Meanwhile, back at the ranch, human beings use the tools that keep us alive and ho hum those that don't. The show really doesn't matter. In the end, Life itself has existed for billions of years and has adapted to change of all kinds. Sure, these boys say they are better than stupid nature. They have shouted this in every generation for thousands of years. Well, if humans and their loud boys ever exist for billions of years, they can laugh all they want, but only then. Right now, we are at 300,000 years and counting down. Endless hubris for the prancing few (got to have that pale penis friends, got to), or loss of a planet and foodmakers to support all this fun? We will indeed see how this goes.
Subscribed because of this brilliant comment. A+.
Thank you. AI and human consciousness are on many minds these days. So is monetization of AI/human consciousness products. Youtube is rife with them. We care about these topics. Makes sense.
Alberto writes, " We know that money rules the company (OpenAI), as it couldn’t be otherwise in a capitalist society."
In theory at least, the reach for AGI could have been funded by public money, just as the Manhattan Project was. The democratic process could have been used to allow the public to have a say in how much of their money should go to AGI development, how fast that development should proceed etc. One way to look at this is that OpenAI had to partner with Microsoft because they had failed to sell the public on their mission.
And, the more I think about it, if AGI is really as revolutionary as claimed, why isn't the government more deeply involved? Why aren't they running the show, just as they did with nuclear weapons? Sure, Microsoft has a lot of money. The U.S. government has far more. Isn't it odd that the U.S. government would let a small room full of private citizens develop, own and control what is proposed to be the most important invention in human history?
A couple of things. OpenAI never wanted public money. First Musk's and the Microsoft's. Second, who says the government isn't involved? The government is always involved to some degree. Surely intelligence agencies. We just don't know!
If AGI is the revolutionary technology that OpenAI seems to claim it will be... Who gives a shit what OpenAI wants? Should the government allow companies to mass produce nukes too?
The government(s) do have a growing interest in regulating the development of AI, but that's different than being fully in charge, as they were with nukes.
I guess this depends to a large degree on how revolutionary AI proves to be. If it's just chatbots, then yea, let the market sort it out. If it's AGI as it's often described, different story.
What doesn't matter is what people not building AGI want lol. You asked why they took Microsoft's money but it's not because they failed to sell to the public, it's because they didn't want to. I agree with your last point, though.
Right, agreed, what doesn't matter is what the people not building AGI want.
Doesn't that pretty much undermine OpenAI's claim that this project is for the benefit of all humanity?
Doesn't the fact that OpenAI chose to partner with Microsoft, instead of with we the public (who have far more money) tell us what the mission of OpenAI really is?
Is Altman's real role that of the charismatic poster child front man in charge of selling us the cover story?
Exactly right. That has been my main criticism of them since the very beginning in 2017-2018 when they were still focused on RL and not on LLMs. You can build your thing, fine, but don't sell it for what it is not.
Neoliberalism is the pense unique as the French say. The only game in town. The religion that says the market and private sector do everything better. Even public sector workers have drank the bong water at this point.
Agree - the problems started before the overnight OpenAI story.
For example, Altman goes to war against Toner regarding a paper she published - claiming that it is negative towards OpenAI. (not really and that's her job anyway).
Altman and Brockman demoted Sutskever about a month ago, and Sutskever had to get the rest of the Board to put him back.
Also, Altman is out fundraising for at least two other companies he's starting. Conflict of interest?? Lack of communication?? Most likely.
Ultimately, money won.
Congrats for a well articulated piece. And also thanks for the unsolicited advices to OpenAi, but hopefully advice for all those out there that are using moral grounds and a not verifiable future in their narrative.
OMG!!! Travis Kelce and Taylor Swift broke up!?!?! Like, WTF?!?!? X-risk isn’t important anymore??? This totes does NOT slap
Wait... OMG!!!! Tay-Tay and Kelce are back together again?!?!? Like, AWESOME!!! Now we DON’T have to worry about x-risk anymore... ever again... because, like, the cray-cray drama has been contained, which TOTES means NO X-RISK HERE! NOTHING TO SEE! MOVE ALONG!
This whole melodrama may be the dumbest and most pointless news story since the live coverage of CHiPs in pursuit of OJ Simpson, bearing down on him at the breakneck speed of ~35 mph.
Has anyone’s p(doom) changed much in the last five days? Just curious.
What has surely changed, and thank god, is people's view of effective altruism and their obsession with x risk (FWIW I dislike just as much the e/acc).
Thanks for keeping us up to speed this week. Interesting, insightful and well written as always. Can you do a newsletter on international affairs as well please?