"The calmness of being able to outsource our trust." - Should have been the title. I am going to have to re-read this when I have more time. So many gems.
The fact that OpenAI explicitly said the scaling laws apply to Sora.
This particular comparison (GPT-3 -> ChatGPT/GPT-4) is more of a rhetorical device to drive my point home. Sora -> Sora 2 is, literally speaking, comparable to nothing.
Alright, makes sense. To me it seems like a strong claim to make. That being said, I’m with you on everything else that you lay out in your piece, it’s pretty scary tech.
One more thing. With regards to uncertainty. The argument I’m leaning towards is that of Harari (I’m pretty sure you’re be familiar with it) in which he explains that democracy is a conversation between informed citizens. When that conversation breaks down, due to trust issues, democracy hangs in the balance.
It ties into your line about: Why be afraid when that was all we knew up until the 19th century? I think we do need to worry about the trust flip, especially when you consider that in pre-20th century there were no large scale democracies.
Only when television and radio entered the picture (pun intented), we were able to organize our societies in a more centralized and egalitarean way. I feel we stand to lose much more from this trust flip than just our sense of certainty.
Democracy in ancient Greece was better than ours. I don't think TV and radio were good for democracy but instead for unifying the narratives to align them with the interests of those behind the media.
Trust issues don't exist just because of the trust flip, but because of the tragedy of commonsense morality - we are different people, with different cultures and beliefs trying to share one space.
Anyway, I agree effects on democracy are complex (e.g. elections) but let's not pretend without AI we'd have perfectly fine democratic systems.
Blockchain technology will allow us to cryptographically verify the provenance of any media. For example, images/videos taken from a camera device will be cryptographically-signed by that device and this proof of provenance will be stored on the blockchain. No amount of AI-horsepower will be able to break the cryptographic proof.
Obviously this will be extremely helpful in a world where there will be a lot of AI-generated fake content.
RE: "We've been here before" with regard to "verify THEN trust"
One has to wonder whether a baseline low-trust society can functionally operate in even the admittedly flawed manner we have come to expect of liberal democratic societies. Feels like a baseline level of trust is required before people expect their fellow men to responsibly navigate liberal democracy. What is the correlate tale of trust receding while general democratic modes increase?
Sure, we've been in low-trust states as a society before, but I'd argue that many Western democracies had only started to fully serve all citizens fairly once that low-trust state receded. To put it another way, it might not END democracy, but it seems assured to erode the progress we've made in putting it into action.
Conspiracy theorist here. I feel like the timing and lack of early regulation around these tools is purposefully designed to align with the 2024 federal election and throw it into chaos. Much like how we're seeing Bitcoin mining and exchanges now getting regulated, only after massive holding companies have bought up most of it, AI generation is being allowed to run rampant and do its damage in the news cycles today, so that it can be reigned in and controlled tomorrow.
except that there's scant evidence that this has actually occurred, so far.
Instead, what we have are warnings about its possible impending impacts. Which I view as a good thing, no matter the source. I think the video disinformation wave will hit eventually, but as far as the election, I think there are probably well-funded efforts to head off the most obviously targeted efforts as the pass.
The real problem is deeper.
I've been harping- I mean, constructively trolling- about the lack of immunity to logical fallacies and language poison in the citizenry for around a quarter-century, at this point. A problem that long preceded the new capabilities of visual and video fakery.
Too few citizens have competence with debate, informal logic and logical fallacy detection, which is the cornerstone of informed skepticism. (I'm mostly familiar with the US, where the lack of those skills is rampant.) It simply isn't taught in American schools. The potential political implications are apparently too sensitive. As a result, most of the rhetoric about "prizing democracy" is cheap talk lip service, not backed up by an informed social consensus with the skill set required to grapple with the harder questions. "Critical thinking" is another phrase that's commonly used by people as a coded synonym for sharing epistemic closure with like-minded others who dwell in the same silo of stovepiped information- even in the case of people with "advanced degrees", who should know better.
Another component of the problem is the naive default to "seeing is believing", which is more or less the problem Kevin Kelly has described. Considered as an aggregated norm, around 70% of human sensory perception faculties is devoted to the visual sense. Without distortion-canceling critical feedback loops in place, people are sitting ducks for over-valuing every video-mediated input from porn to TV commercials and political ads.
"The camera doesn't lie" and "one picture is worth 1000 words" are two more examples of the horseshit foisted on the gullible. That sort of fatuity is what led to the replacement of reading books in high school with Watching The Movie. The TV Generation shift away from literacy has been in effect for around 60 years, and as a result, Aliteracy is practically universal. Reading books is a niche thing, even among "elites." Whereas watching television and popular movies (and more recently videogames, and streaming) is so ubiquitous that those media now account for the vast majority of conversational metaphors in popular culture. Radio- which lacks preformatted image and symbolic content- has been demoted practically to a superfluous medium, even for music (in part because musical content is practically perceived as outright competition for advertising content- unless it's name-checking consumer brands, of course.)
A/V Media Rules, and it's ruled since the 1980s, with the advent of MTV, when TV ads first got their "youth market pitch" down. Then we got CNN, which made its bones with the eerie spectacle of the Persian Gulf War.
I became especially sensitive to these transformation as a result of ditching my television in 1981. I stayed away from it for around the next two decades- but most of the educational value resulted from my first 12 months of avoiding it. One year away, and I never looked at TV the same way again. Not that it's a question that crops up very often, but- the next time you rear someone explaining that they don't read books because they "don't have the time", check how much TV or video they watch. We're living in a Zeitgeist where reading 1-3 non-fiction books a year qualifies someone as "well-read", even if their reading habits consist exclusively of uncritically swallowing partisan dreck by ignoramuses. Often, reading material of that sort is (ghost)written by TV News Celebrities, and more or less supplemental to their TV appearances, where the real action is.
The absence of logical fallacy detection skills and Aliteracy are what's really doing the damage to democracy. Mark my words. One of the most common manifestations is the eagerness to jump to conclusions, and I notice it across the political and ideological spectrum. A lack of resistance to suggestion that leaves people as easy marks for fakery by unscrupulous actors and swindlers.
I can agree with you on that. Add to it hoaxes and psyops in an age where people just scroll past headlines and don't have a coherent worldview to begin with. Schools are designed to make people stupid. The unnatural decline in intelligence is being met by an unnatural manipulation. There's far too much at stake to let the average citizen figure out what's going on in the real world.
Manipulation is abundant, but most of the assent to it is due to a failure to examine assumptions. I don't think the manipulators are all that dialed in to some ultra-crafty grand ambition, either. What I notice is the pettiness of all those private ends that they imagine to justify their unspeakable means. It's like they never had another thought outside of the rut they're in. They don't even seem to have asked "what for"?
If the blind lead the blind, both will fall into a pit.
If you're familiar with H.G. Wells, Merilyn Ferguson, and the Open Conspiracy concept, it doesn't need to be ultra-crafty to be grand and ongoing. A smart populace could resist it, and individuals are responsible for themselves in theory, but the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best.
"The calmness of being able to outsource our trust." - Should have been the title. I am going to have to re-read this when I have more time. So many gems.
Thank you! Let me know which ones you like
I’ve heard more people call Sora the GPT3 moment for text-to-video. “We can expect the Sora → Sora 2 leap to be comparable to GPT-3 → ChatGPT”
What makes you assume that?
The fact that OpenAI explicitly said the scaling laws apply to Sora.
This particular comparison (GPT-3 -> ChatGPT/GPT-4) is more of a rhetorical device to drive my point home. Sora -> Sora 2 is, literally speaking, comparable to nothing.
Alright, makes sense. To me it seems like a strong claim to make. That being said, I’m with you on everything else that you lay out in your piece, it’s pretty scary tech.
One more thing. With regards to uncertainty. The argument I’m leaning towards is that of Harari (I’m pretty sure you’re be familiar with it) in which he explains that democracy is a conversation between informed citizens. When that conversation breaks down, due to trust issues, democracy hangs in the balance.
It ties into your line about: Why be afraid when that was all we knew up until the 19th century? I think we do need to worry about the trust flip, especially when you consider that in pre-20th century there were no large scale democracies.
Only when television and radio entered the picture (pun intented), we were able to organize our societies in a more centralized and egalitarean way. I feel we stand to lose much more from this trust flip than just our sense of certainty.
Democracy in ancient Greece was better than ours. I don't think TV and radio were good for democracy but instead for unifying the narratives to align them with the interests of those behind the media.
Trust issues don't exist just because of the trust flip, but because of the tragedy of commonsense morality - we are different people, with different cultures and beliefs trying to share one space.
Anyway, I agree effects on democracy are complex (e.g. elections) but let's not pretend without AI we'd have perfectly fine democratic systems.
Blockchain technology will allow us to cryptographically verify the provenance of any media. For example, images/videos taken from a camera device will be cryptographically-signed by that device and this proof of provenance will be stored on the blockchain. No amount of AI-horsepower will be able to break the cryptographic proof.
Obviously this will be extremely helpful in a world where there will be a lot of AI-generated fake content.
RE: "We've been here before" with regard to "verify THEN trust"
One has to wonder whether a baseline low-trust society can functionally operate in even the admittedly flawed manner we have come to expect of liberal democratic societies. Feels like a baseline level of trust is required before people expect their fellow men to responsibly navigate liberal democracy. What is the correlate tale of trust receding while general democratic modes increase?
Sure, we've been in low-trust states as a society before, but I'd argue that many Western democracies had only started to fully serve all citizens fairly once that low-trust state receded. To put it another way, it might not END democracy, but it seems assured to erode the progress we've made in putting it into action.
Conspiracy theorist here. I feel like the timing and lack of early regulation around these tools is purposefully designed to align with the 2024 federal election and throw it into chaos. Much like how we're seeing Bitcoin mining and exchanges now getting regulated, only after massive holding companies have bought up most of it, AI generation is being allowed to run rampant and do its damage in the news cycles today, so that it can be reigned in and controlled tomorrow.
except that there's scant evidence that this has actually occurred, so far.
Instead, what we have are warnings about its possible impending impacts. Which I view as a good thing, no matter the source. I think the video disinformation wave will hit eventually, but as far as the election, I think there are probably well-funded efforts to head off the most obviously targeted efforts as the pass.
The real problem is deeper.
I've been harping- I mean, constructively trolling- about the lack of immunity to logical fallacies and language poison in the citizenry for around a quarter-century, at this point. A problem that long preceded the new capabilities of visual and video fakery.
Too few citizens have competence with debate, informal logic and logical fallacy detection, which is the cornerstone of informed skepticism. (I'm mostly familiar with the US, where the lack of those skills is rampant.) It simply isn't taught in American schools. The potential political implications are apparently too sensitive. As a result, most of the rhetoric about "prizing democracy" is cheap talk lip service, not backed up by an informed social consensus with the skill set required to grapple with the harder questions. "Critical thinking" is another phrase that's commonly used by people as a coded synonym for sharing epistemic closure with like-minded others who dwell in the same silo of stovepiped information- even in the case of people with "advanced degrees", who should know better.
Another component of the problem is the naive default to "seeing is believing", which is more or less the problem Kevin Kelly has described. Considered as an aggregated norm, around 70% of human sensory perception faculties is devoted to the visual sense. Without distortion-canceling critical feedback loops in place, people are sitting ducks for over-valuing every video-mediated input from porn to TV commercials and political ads.
"The camera doesn't lie" and "one picture is worth 1000 words" are two more examples of the horseshit foisted on the gullible. That sort of fatuity is what led to the replacement of reading books in high school with Watching The Movie. The TV Generation shift away from literacy has been in effect for around 60 years, and as a result, Aliteracy is practically universal. Reading books is a niche thing, even among "elites." Whereas watching television and popular movies (and more recently videogames, and streaming) is so ubiquitous that those media now account for the vast majority of conversational metaphors in popular culture. Radio- which lacks preformatted image and symbolic content- has been demoted practically to a superfluous medium, even for music (in part because musical content is practically perceived as outright competition for advertising content- unless it's name-checking consumer brands, of course.)
A/V Media Rules, and it's ruled since the 1980s, with the advent of MTV, when TV ads first got their "youth market pitch" down. Then we got CNN, which made its bones with the eerie spectacle of the Persian Gulf War.
I became especially sensitive to these transformation as a result of ditching my television in 1981. I stayed away from it for around the next two decades- but most of the educational value resulted from my first 12 months of avoiding it. One year away, and I never looked at TV the same way again. Not that it's a question that crops up very often, but- the next time you rear someone explaining that they don't read books because they "don't have the time", check how much TV or video they watch. We're living in a Zeitgeist where reading 1-3 non-fiction books a year qualifies someone as "well-read", even if their reading habits consist exclusively of uncritically swallowing partisan dreck by ignoramuses. Often, reading material of that sort is (ghost)written by TV News Celebrities, and more or less supplemental to their TV appearances, where the real action is.
The absence of logical fallacy detection skills and Aliteracy are what's really doing the damage to democracy. Mark my words. One of the most common manifestations is the eagerness to jump to conclusions, and I notice it across the political and ideological spectrum. A lack of resistance to suggestion that leaves people as easy marks for fakery by unscrupulous actors and swindlers.
I can agree with you on that. Add to it hoaxes and psyops in an age where people just scroll past headlines and don't have a coherent worldview to begin with. Schools are designed to make people stupid. The unnatural decline in intelligence is being met by an unnatural manipulation. There's far too much at stake to let the average citizen figure out what's going on in the real world.
Manipulation is abundant, but most of the assent to it is due to a failure to examine assumptions. I don't think the manipulators are all that dialed in to some ultra-crafty grand ambition, either. What I notice is the pettiness of all those private ends that they imagine to justify their unspeakable means. It's like they never had another thought outside of the rut they're in. They don't even seem to have asked "what for"?
If the blind lead the blind, both will fall into a pit.
If you're familiar with H.G. Wells, Merilyn Ferguson, and the Open Conspiracy concept, it doesn't need to be ultra-crafty to be grand and ongoing. A smart populace could resist it, and individuals are responsible for themselves in theory, but the technology of social engineering keeps multiplying, while self-education and discipline has progressed in a linear scale at best.