"The calmness of being able to outsource our trust." - Should have been the title. I am going to have to re-read this when I have more time. So many gems.
The fact that OpenAI explicitly said the scaling laws apply to Sora.
This particular comparison (GPT-3 -> ChatGPT/GPT-4) is more of a rhetorical device to drive my point home. Sora -> Sora 2 is, literally speaking, comparable to nothing.
Alright, makes sense. To me it seems like a strong claim to make. That being said, I’m with you on everything else that you lay out in your piece, it’s pretty scary tech.
One more thing. With regards to uncertainty. The argument I’m leaning towards is that of Harari (I’m pretty sure you’re be familiar with it) in which he explains that democracy is a conversation between informed citizens. When that conversation breaks down, due to trust issues, democracy hangs in the balance.
It ties into your line about: Why be afraid when that was all we knew up until the 19th century? I think we do need to worry about the trust flip, especially when you consider that in pre-20th century there were no large scale democracies.
Only when television and radio entered the picture (pun intented), we were able to organize our societies in a more centralized and egalitarean way. I feel we stand to lose much more from this trust flip than just our sense of certainty.
Democracy in ancient Greece was better than ours. I don't think TV and radio were good for democracy but instead for unifying the narratives to align them with the interests of those behind the media.
Trust issues don't exist just because of the trust flip, but because of the tragedy of commonsense morality - we are different people, with different cultures and beliefs trying to share one space.
Anyway, I agree effects on democracy are complex (e.g. elections) but let's not pretend without AI we'd have perfectly fine democratic systems.
You do know that women and slaves had no rights, right? It was a direct democacy, not representative, and limited to the city Athens. ... I guarantee you democracy in ancient Athens was not better than ours.
Also, TV and radio are literally the reason large scale democracies like to today exist:
That women and slaves didn't have rights was independent of how democracy was established. Of course, that's worse but has nothing to do with how democracy works. It's a (terrible) social decision that can exist or not independently.
The reason democracy exists is the enlightenment first and foremost, not TV lol (whatever Harari says, let's not go into the "correlation is not causation" thing). Thinking, not watching TV is good for democracy.
I don't think AI can degrade democracy the way you say except as a downstream consequence of losing certainty or knowing what/who to trust. It can have an effect on elections though.
Blockchain technology will allow us to cryptographically verify the provenance of any media. For example, images/videos taken from a camera device will be cryptographically-signed by that device and this proof of provenance will be stored on the blockchain. No amount of AI-horsepower will be able to break the cryptographic proof.
Obviously this will be extremely helpful in a world where there will be a lot of AI-generated fake content.
RE: "We've been here before" with regard to "verify THEN trust"
One has to wonder whether a baseline low-trust society can functionally operate in even the admittedly flawed manner we have come to expect of liberal democratic societies. Feels like a baseline level of trust is required before people expect their fellow men to responsibly navigate liberal democracy. What is the correlate tale of trust receding while general democratic modes increase?
Sure, we've been in low-trust states as a society before, but I'd argue that many Western democracies had only started to fully serve all citizens fairly once that low-trust state receded. To put it another way, it might not END democracy, but it seems assured to erode the progress we've made in putting it into action.
Conspiracy theorist here. I feel like the timing and lack of early regulation around these tools is purposefully designed to align with the 2024 federal election and throw it into chaos. Much like how we're seeing Bitcoin mining and exchanges now getting regulated, only after massive holding companies have bought up most of it, AI generation is being allowed to run rampant and do its damage in the news cycles today, so that it can be reigned in and controlled tomorrow.
except that there's scant evidence that this has actually occurred, so far.
Instead, what we have are warnings about its possible impending impacts. Which I view as a good thing, no matter the source. I think the video disinformation wave will hit eventually, but as far as the election, I think there are probably well-funded efforts to head off the most obviously targeted efforts as the pass.
The real problem is deeper.
I've been harping- I mean, constructively trolling- about the lack of immunity to logical fallacies and language poison in the citizenry for around a quarter-century, at this point. A problem that long preceded the new capabilities of visual and video fakery.
Too few citizens have competence with debate, informal logic and logical fallacy detection, which is the cornerstone of informed skepticism. (I'm mostly familiar with the US, where the lack of those skills is rampant.) It simply isn't taught in American schools. The potential political implications are apparently too sensitive. As a result, most of the rhetoric about "prizing democracy" is cheap talk lip service, not backed up by an informed social consensus with the skill set required to grapple with the harder questions. "Critical thinking" is another phrase that's commonly used by people as a coded synonym for sharing epistemic closure with like-minded others who dwell in the same silo of stovepiped information- even in the case of people with "advanced degrees", who should know better.
Another component of the problem is the naive default to "seeing is believing", which is more or less the problem Kevin Kelly has described. Considered as an aggregated norm, around 70% of human sensory perception faculties is devoted to the visual sense. Without distortion-canceling critical feedback loops in place, people are sitting ducks for over-valuing every video-mediated input from porn to TV commercials and political ads.
"The camera doesn't lie" and "one picture is worth 1000 words" are two more examples of the horseshit foisted on the gullible. That sort of fatuity is what led to the replacement of reading books in high school with Watching The Movie. The TV Generation shift away from literacy has been in effect for around 60 years, and as a result, Aliteracy is practically universal. Reading books is a niche thing, even among "elites." Whereas watching television and popular movies (and more recently videogames, and streaming) is so ubiquitous that those media now account for the vast majority of conversational metaphors in popular culture. Radio- which lacks preformatted image and symbolic content- has been demoted practically to a superfluous medium, even for music (in part because musical content is practically perceived as outright competition for advertising content- unless it's name-checking consumer brands, of course.)
A/V Media Rules, and it's ruled since the 1980s, with the advent of MTV, when TV ads first got their "youth market pitch" down. Then we got CNN, which made its bones with the eerie spectacle of the Persian Gulf War.
I became especially sensitive to these transformation as a result of ditching my television in 1981. I stayed away from it for around the next two decades- but most of the educational value resulted from my first 12 months of avoiding it. One year away, and I never looked at TV the same way again. Not that it's a question that crops up very often, but- the next time you rear someone explaining that they don't read books because they "don't have the time", check how much TV or video they watch. We're living in a Zeitgeist where reading 1-3 non-fiction books a year qualifies someone as "well-read", even if their reading habits consist exclusively of uncritically swallowing partisan dreck by ignoramuses. Often, reading material of that sort is (ghost)written by TV News Celebrities, and more or less supplemental to their TV appearances, where the real action is.
The absence of logical fallacy detection skills and Aliteracy are what's really doing the damage to democracy. Mark my words. One of the most common manifestations is the eagerness to jump to conclusions, and I notice it across the political and ideological spectrum. A lack of resistance to suggestion that leaves people as easy marks for fakery by unscrupulous actors and swindlers.
I can agree with you on that. Add to it hoaxes and psyops in an age where people just scroll past headlines and don't have a coherent worldview to begin with. Schools are designed to make people stupid. The unnatural decline in intelligence is being met by an unnatural manipulation. There's far too much at stake to let the average citizen figure out what's going on in the real world.
Manipulation is abundant, but most of the assent to it is due to a failure to examine assumptions. I don't think the manipulators are all that dialed in to some ultra-crafty grand ambition, either. What I notice is the pettiness of all those private ends that they imagine to justify their unspeakable means. It's like they never had another thought outside of the rut they're in. They don't even seem to have asked "what for"?
If the blind lead the blind, both will fall into a pit.
If you're familiar with H.G. Wells, Merilyn Ferguson, and the Open Conspiracy concept, it doesn't need to be ultra-crafty to be grand and ongoing. A smart populace could resist it, and individuals are responsible for themselves in theory, but the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best.
I don't think about my society in grand monolithic terms, or draw firm sweeping conclusions about what's happening in the multifarious array of phenomena commonly referred to as "society". It's pretentious. Other people are going to do what they're going to do. I'm responsible for myself, in practice.
What you're describing as Open Conspiracy- "it"- is probably best described as something entirely different. Not everyone is on the same page, to understate the case. That's why autocratic leaders are the most paranoid about being deposed. There's always a palace intrigue behind the scenes. The more absolute the rule, the more the conniving. Selfish, power-hungry people have their private agendas first and foremost. Conspiracies of one, so to speak. Running around behind each others backs. For such people, all alliances- conspiratorial or otherwise- are transactional, and contingent on short-term advantage.
I'm familiar with conspiracies- as a history researcher, not a grand unified theorist. They're more like the monkey wrenches in the works of Power than they are its impelling motive. Not that this makes earthly Power- lawful-monopoly-on-force Power, some great thing in itself. But that Power asserts its aims openly, even if the assumptions are typically unexamined and often not especially rational or responsive to the ostensible goals being pursued. The unconscious short-term selfish motivations of humans are entirely capable of drowning us in our own shit, as robotic as yeast manufacturing ethyl alcohol from consuming sugar and lethal toxifying their habitat until the entire yeast colony dies from mass poisoning.
To pivot to the problem of conscious responses formulated to contend with problems in human society; they are not immune to folly, either. Spotlessly noble intentions are not a guarantor of beneficial results in that regard. Alcohol isn't the doom of humanity that it is to yeast, but drunkenness and alcoholism are indisputably social ills. Alcohol Prohibition in the US wasn't put in place as a Machiavellian scheme by a small group of unscrupulous profiteers meeting in secret, it was openly advocated and implemented by sincere social reformers who took the idea entirely too far. A massive number of conspiracies and corruption (all corruption has an element of conspiracy) followed in its wake. The conspiracies took all sorts of forms; some of them were quite large. None of them would have existed without the foolhardy attempt to reduce the harms of alcohol with a punitive moralist popular movement to ban it entirely by criminalizing the trade.
The War on Drugs is a couple of orders of magnitude worse, and it's basically been in place for 100 years, whereas Prohibition was only in place for 13 years. The longer the War on Drugs goes on, the harder it's going to be to emerge from the quagmire of the societal fallout. But as with Alcohol Prohibition, the War On Drugs was openly advocated for, initiated, and implemented by sincere social reformers. It had wide popular support across the board, with the only real divide appearing--after some decades--on the basis of age demographics.
Now we're at a point where there have been a few moves to liberalize the drug laws- and the War On Drugs advocates are accusing the entire drug law reform movement of having been a conspiracy! It's undeniable that there are people with dollar signs in their eyes looking to cash in on commercial cannabis markets, but it's being done quite openly. The legislators need to do their jobs and stop giving them a blank check.
I realize that there are legislators being influenced by Big Corporate Cannabis through campaign contributions, for example. But that's a much wider problem, and much more accurately diagnosed by Marxian structural analysis than by inchoate conspiracism. (I'm not a Marxist, but I'm able to hear out any ideological gloss that makes a solid, evidence-based case. And in some ways, as a social diagnostician, Marx was on to something.)
Observing the conspiracist extremes at both ends, it's like watching social networks stampeding over rumors of the sky falling. Neither grand unified conspiracy theory narrative holds up. Instead, what we have are middleman conspiracies. Lots of tentacles, but not attached to one big octopus. (A least not as a human social network. The default to the same set of drives and desires is the same, for those engaging in conspiracy for egoistic advantage instead of out of dire necessity.)
"the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best."
That's a subjective estimation, not the objective truth you imply. I don't even try to turn my impressions into such a calculation.
It isn't just "social engineering" technologies that are proliferating, in the present age; the Internet has enabled and empowered so many social phenomena that it doesn't make sense to consider it only in terms of "social engineering" aspects. At its farthest fringe, the Internet is an anarchic frontier. The "social engineering" technologies are more or less attempts to play catch-up. I think the use of AI for preemptory purposes is a terrible misstep, because all efforts at pervasive censorship inspire opposition and resistance. Sabotage, subversion, satire. The most powerful Internet censorship regime in the world in in China, but I suspect they've bitten off more than they can chew.
In that regard, AI may prove to be more of a liberating technology than a means of manipulation. The rigidity of closed societies leaves them vulnerable to being undermined by the introduction of chaotic aspects into the system. Open societies have more checks and balances to contend with the sort of information anomalies that ChatGPT and DALL-E are able to generate so effortlessly. If I was in AI, I'd be working on a pinyin Chinese version.
"The calmness of being able to outsource our trust." - Should have been the title. I am going to have to re-read this when I have more time. So many gems.
Thank you! Let me know which ones you like
I’ve heard more people call Sora the GPT3 moment for text-to-video. “We can expect the Sora → Sora 2 leap to be comparable to GPT-3 → ChatGPT”
What makes you assume that?
The fact that OpenAI explicitly said the scaling laws apply to Sora.
This particular comparison (GPT-3 -> ChatGPT/GPT-4) is more of a rhetorical device to drive my point home. Sora -> Sora 2 is, literally speaking, comparable to nothing.
Alright, makes sense. To me it seems like a strong claim to make. That being said, I’m with you on everything else that you lay out in your piece, it’s pretty scary tech.
One more thing. With regards to uncertainty. The argument I’m leaning towards is that of Harari (I’m pretty sure you’re be familiar with it) in which he explains that democracy is a conversation between informed citizens. When that conversation breaks down, due to trust issues, democracy hangs in the balance.
It ties into your line about: Why be afraid when that was all we knew up until the 19th century? I think we do need to worry about the trust flip, especially when you consider that in pre-20th century there were no large scale democracies.
Only when television and radio entered the picture (pun intented), we were able to organize our societies in a more centralized and egalitarean way. I feel we stand to lose much more from this trust flip than just our sense of certainty.
Democracy in ancient Greece was better than ours. I don't think TV and radio were good for democracy but instead for unifying the narratives to align them with the interests of those behind the media.
Trust issues don't exist just because of the trust flip, but because of the tragedy of commonsense morality - we are different people, with different cultures and beliefs trying to share one space.
Anyway, I agree effects on democracy are complex (e.g. elections) but let's not pretend without AI we'd have perfectly fine democratic systems.
You do know that women and slaves had no rights, right? It was a direct democacy, not representative, and limited to the city Athens. ... I guarantee you democracy in ancient Athens was not better than ours.
Also, TV and radio are literally the reason large scale democracies like to today exist:
https://youtu.be/7JkPWHr7sTY?feature=shared&t=1085
"but let's not pretend without AI we'd have perfectly fine democratic systems."
- so if it's bad already, we shouldn't worry about further degradation? Got it. 😅
That women and slaves didn't have rights was independent of how democracy was established. Of course, that's worse but has nothing to do with how democracy works. It's a (terrible) social decision that can exist or not independently.
The reason democracy exists is the enlightenment first and foremost, not TV lol (whatever Harari says, let's not go into the "correlation is not causation" thing). Thinking, not watching TV is good for democracy.
I don't think AI can degrade democracy the way you say except as a downstream consequence of losing certainty or knowing what/who to trust. It can have an effect on elections though.
Blockchain technology will allow us to cryptographically verify the provenance of any media. For example, images/videos taken from a camera device will be cryptographically-signed by that device and this proof of provenance will be stored on the blockchain. No amount of AI-horsepower will be able to break the cryptographic proof.
Obviously this will be extremely helpful in a world where there will be a lot of AI-generated fake content.
RE: "We've been here before" with regard to "verify THEN trust"
One has to wonder whether a baseline low-trust society can functionally operate in even the admittedly flawed manner we have come to expect of liberal democratic societies. Feels like a baseline level of trust is required before people expect their fellow men to responsibly navigate liberal democracy. What is the correlate tale of trust receding while general democratic modes increase?
Sure, we've been in low-trust states as a society before, but I'd argue that many Western democracies had only started to fully serve all citizens fairly once that low-trust state receded. To put it another way, it might not END democracy, but it seems assured to erode the progress we've made in putting it into action.
Conspiracy theorist here. I feel like the timing and lack of early regulation around these tools is purposefully designed to align with the 2024 federal election and throw it into chaos. Much like how we're seeing Bitcoin mining and exchanges now getting regulated, only after massive holding companies have bought up most of it, AI generation is being allowed to run rampant and do its damage in the news cycles today, so that it can be reigned in and controlled tomorrow.
except that there's scant evidence that this has actually occurred, so far.
Instead, what we have are warnings about its possible impending impacts. Which I view as a good thing, no matter the source. I think the video disinformation wave will hit eventually, but as far as the election, I think there are probably well-funded efforts to head off the most obviously targeted efforts as the pass.
The real problem is deeper.
I've been harping- I mean, constructively trolling- about the lack of immunity to logical fallacies and language poison in the citizenry for around a quarter-century, at this point. A problem that long preceded the new capabilities of visual and video fakery.
Too few citizens have competence with debate, informal logic and logical fallacy detection, which is the cornerstone of informed skepticism. (I'm mostly familiar with the US, where the lack of those skills is rampant.) It simply isn't taught in American schools. The potential political implications are apparently too sensitive. As a result, most of the rhetoric about "prizing democracy" is cheap talk lip service, not backed up by an informed social consensus with the skill set required to grapple with the harder questions. "Critical thinking" is another phrase that's commonly used by people as a coded synonym for sharing epistemic closure with like-minded others who dwell in the same silo of stovepiped information- even in the case of people with "advanced degrees", who should know better.
Another component of the problem is the naive default to "seeing is believing", which is more or less the problem Kevin Kelly has described. Considered as an aggregated norm, around 70% of human sensory perception faculties is devoted to the visual sense. Without distortion-canceling critical feedback loops in place, people are sitting ducks for over-valuing every video-mediated input from porn to TV commercials and political ads.
"The camera doesn't lie" and "one picture is worth 1000 words" are two more examples of the horseshit foisted on the gullible. That sort of fatuity is what led to the replacement of reading books in high school with Watching The Movie. The TV Generation shift away from literacy has been in effect for around 60 years, and as a result, Aliteracy is practically universal. Reading books is a niche thing, even among "elites." Whereas watching television and popular movies (and more recently videogames, and streaming) is so ubiquitous that those media now account for the vast majority of conversational metaphors in popular culture. Radio- which lacks preformatted image and symbolic content- has been demoted practically to a superfluous medium, even for music (in part because musical content is practically perceived as outright competition for advertising content- unless it's name-checking consumer brands, of course.)
A/V Media Rules, and it's ruled since the 1980s, with the advent of MTV, when TV ads first got their "youth market pitch" down. Then we got CNN, which made its bones with the eerie spectacle of the Persian Gulf War.
I became especially sensitive to these transformation as a result of ditching my television in 1981. I stayed away from it for around the next two decades- but most of the educational value resulted from my first 12 months of avoiding it. One year away, and I never looked at TV the same way again. Not that it's a question that crops up very often, but- the next time you rear someone explaining that they don't read books because they "don't have the time", check how much TV or video they watch. We're living in a Zeitgeist where reading 1-3 non-fiction books a year qualifies someone as "well-read", even if their reading habits consist exclusively of uncritically swallowing partisan dreck by ignoramuses. Often, reading material of that sort is (ghost)written by TV News Celebrities, and more or less supplemental to their TV appearances, where the real action is.
The absence of logical fallacy detection skills and Aliteracy are what's really doing the damage to democracy. Mark my words. One of the most common manifestations is the eagerness to jump to conclusions, and I notice it across the political and ideological spectrum. A lack of resistance to suggestion that leaves people as easy marks for fakery by unscrupulous actors and swindlers.
I can agree with you on that. Add to it hoaxes and psyops in an age where people just scroll past headlines and don't have a coherent worldview to begin with. Schools are designed to make people stupid. The unnatural decline in intelligence is being met by an unnatural manipulation. There's far too much at stake to let the average citizen figure out what's going on in the real world.
Manipulation is abundant, but most of the assent to it is due to a failure to examine assumptions. I don't think the manipulators are all that dialed in to some ultra-crafty grand ambition, either. What I notice is the pettiness of all those private ends that they imagine to justify their unspeakable means. It's like they never had another thought outside of the rut they're in. They don't even seem to have asked "what for"?
If the blind lead the blind, both will fall into a pit.
If you're familiar with H.G. Wells, Merilyn Ferguson, and the Open Conspiracy concept, it doesn't need to be ultra-crafty to be grand and ongoing. A smart populace could resist it, and individuals are responsible for themselves in theory, but the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best.
I don't think about my society in grand monolithic terms, or draw firm sweeping conclusions about what's happening in the multifarious array of phenomena commonly referred to as "society". It's pretentious. Other people are going to do what they're going to do. I'm responsible for myself, in practice.
What you're describing as Open Conspiracy- "it"- is probably best described as something entirely different. Not everyone is on the same page, to understate the case. That's why autocratic leaders are the most paranoid about being deposed. There's always a palace intrigue behind the scenes. The more absolute the rule, the more the conniving. Selfish, power-hungry people have their private agendas first and foremost. Conspiracies of one, so to speak. Running around behind each others backs. For such people, all alliances- conspiratorial or otherwise- are transactional, and contingent on short-term advantage.
I'm familiar with conspiracies- as a history researcher, not a grand unified theorist. They're more like the monkey wrenches in the works of Power than they are its impelling motive. Not that this makes earthly Power- lawful-monopoly-on-force Power, some great thing in itself. But that Power asserts its aims openly, even if the assumptions are typically unexamined and often not especially rational or responsive to the ostensible goals being pursued. The unconscious short-term selfish motivations of humans are entirely capable of drowning us in our own shit, as robotic as yeast manufacturing ethyl alcohol from consuming sugar and lethal toxifying their habitat until the entire yeast colony dies from mass poisoning.
To pivot to the problem of conscious responses formulated to contend with problems in human society; they are not immune to folly, either. Spotlessly noble intentions are not a guarantor of beneficial results in that regard. Alcohol isn't the doom of humanity that it is to yeast, but drunkenness and alcoholism are indisputably social ills. Alcohol Prohibition in the US wasn't put in place as a Machiavellian scheme by a small group of unscrupulous profiteers meeting in secret, it was openly advocated and implemented by sincere social reformers who took the idea entirely too far. A massive number of conspiracies and corruption (all corruption has an element of conspiracy) followed in its wake. The conspiracies took all sorts of forms; some of them were quite large. None of them would have existed without the foolhardy attempt to reduce the harms of alcohol with a punitive moralist popular movement to ban it entirely by criminalizing the trade.
The War on Drugs is a couple of orders of magnitude worse, and it's basically been in place for 100 years, whereas Prohibition was only in place for 13 years. The longer the War on Drugs goes on, the harder it's going to be to emerge from the quagmire of the societal fallout. But as with Alcohol Prohibition, the War On Drugs was openly advocated for, initiated, and implemented by sincere social reformers. It had wide popular support across the board, with the only real divide appearing--after some decades--on the basis of age demographics.
Now we're at a point where there have been a few moves to liberalize the drug laws- and the War On Drugs advocates are accusing the entire drug law reform movement of having been a conspiracy! It's undeniable that there are people with dollar signs in their eyes looking to cash in on commercial cannabis markets, but it's being done quite openly. The legislators need to do their jobs and stop giving them a blank check.
I realize that there are legislators being influenced by Big Corporate Cannabis through campaign contributions, for example. But that's a much wider problem, and much more accurately diagnosed by Marxian structural analysis than by inchoate conspiracism. (I'm not a Marxist, but I'm able to hear out any ideological gloss that makes a solid, evidence-based case. And in some ways, as a social diagnostician, Marx was on to something.)
Observing the conspiracist extremes at both ends, it's like watching social networks stampeding over rumors of the sky falling. Neither grand unified conspiracy theory narrative holds up. Instead, what we have are middleman conspiracies. Lots of tentacles, but not attached to one big octopus. (A least not as a human social network. The default to the same set of drives and desires is the same, for those engaging in conspiracy for egoistic advantage instead of out of dire necessity.)
"the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best."
That's a subjective estimation, not the objective truth you imply. I don't even try to turn my impressions into such a calculation.
It isn't just "social engineering" technologies that are proliferating, in the present age; the Internet has enabled and empowered so many social phenomena that it doesn't make sense to consider it only in terms of "social engineering" aspects. At its farthest fringe, the Internet is an anarchic frontier. The "social engineering" technologies are more or less attempts to play catch-up. I think the use of AI for preemptory purposes is a terrible misstep, because all efforts at pervasive censorship inspire opposition and resistance. Sabotage, subversion, satire. The most powerful Internet censorship regime in the world in in China, but I suspect they've bitten off more than they can chew.
In that regard, AI may prove to be more of a liberating technology than a means of manipulation. The rigidity of closed societies leaves them vulnerable to being undermined by the introduction of chaotic aspects into the system. Open societies have more checks and balances to contend with the sort of information anomalies that ChatGPT and DALL-E are able to generate so effortlessly. If I was in AI, I'd be working on a pinyin Chinese version.