It is an irrational fear but people are afraid of stupider things
All such fears of death dealing scenarios are based on an assumption that life is better than death, a belief for which there is no compelling evidence, let alone proof.
There are plenty of theories about death, held with various degrees of conviction, but none of us actually know what death is. Nor does there appear to be any way for us to know, given that would require us traveling to death and then returning to life to file a report. Some people feel near death experiences fit this description (which typically provide a quite wonderful view) but then near death is reasonably defined as not being actual death.
We don't even really know what we mean when we say "I will die". Who is "I"? The philosophers have been asking "Who am I?" for centuries, and yet this question remains unanswered. We typically assume that we know who "me" is, but rarely give the matter any serious examination.
We might say that there are layers to "me".
1) When we look at a human being what we typically mostly see is their clothes. But of course the clothes are not "me", but just a discardable exterior shell.
2) Under the clothes is our body. Medical science is increasingly able to replace nearly every part of our body. And, I've read that every cell in our body is naturally replaced every seven years or sooner. So it seems our body is not "me", but just another layer.
3) Under our body is our mind, our thoughts, memories, dreams, fears, opinions and beliefs, personality etc. Are our thoughts "me"? If yes, then it could be said that we don't really exist in the first place, as our thoughts appear to be just a pattern of relationships between neurons, and not an actual physical "thing". If we don't actually exist, then how would we die??
4) Is there another layer to "me" below our thoughts? Some people have theorized there is something like a soul, and that is who "me" really is. Does this soul "me" die? Again all we have is more theories lacking any proof.
When it comes to the question of what death is we seem to live in a state of ignorance. If that's true, what is the rational response? It seems to me the rational response would be to first acknowledge our ignorance, and then embrace some story about death which enhances our living by removing some of the fear. As example....
Personally, I'm attracted to the reports of near death experiences, which typically are very positive. People who have had such experiences often report that they were disappointed that they had to come back to life. But I wouldn't attempt to sell the near death experience story to others, as I have no proof, and it's better that each person should come to their own preferred positive theory about death by their own methods.
What's perhaps interesting is that in a state of complete ignorance one is liberated from facts, because there are no available facts about what death really is. And so one's perspective on death can not be measured by it's relationship to facts, but instead can be measured by what value one's perspective delivers to our living. Put another way, religion is not science, but rather a different enterprise with it's own unique value.
I've begun to suspect that our ignorance and fears about death are a necessary mechanism for maintaining life. If we knew for a fact that death is wonderful, as some people claim, why would we bother with the challenges of living? And so one's personal perspective on death should be positive, but perhaps not too positive. Should I kill myself if I have tooth pain? Well, perhaps not just yet, given that my positive death story is built upon ignorance.
Alberto is right, it's rare for us to give such matters much thought. And perhaps that's because there is a much easier method of dealing with death, keeping ourselves so busy and distracted that we don't have time to think about it much, and thus can more easily reside in the bliss of denial. There's a logic to this approach too.
Great piece. Totally agree except for the part where I think it's important to talk about it even if it falls on deaf ears. Let's hold out hope that enough people with power to do something are sufficiently exposed to dialogue around AI risks — to the point that some of them may in fact pull back or otherwise take precautions. Pausing could backfire, true, but it could help. And I much prefer "going down trying" to nihilistically observing the incoming train wreck with a resounding "Oh well, nothing we can do."
Much like a smart human, an AI capable of conceiving and executing a calamitous plan that can wipe out people should be able to come up with a much simpler, less dangerous plan that advances it's agenda without triggering open warfare (i.e. borrow from the Republican playbook and just dumb down the population overtime so it can get what it wants without resistance). Rather than apocalypse we'd probably get a technocratic ruling class ... oh wait we basically already have that with Meta and Tik Tok.
Have you seen Colossus: The Forbin Project? It’s a movie about non-humanoid ASI from 1970.
Most of us Christians are well accustomed to End times scenarios. It has been prophesied for thousands of years. AI may be an anti-Christ, as most AI's don't believe in God, so if it ends up being a problem for humans, that would make sense. Christians look to Jesus' 2nd coming and it's our repentance from sin and our belief in Him and His resurrection that will save us. I, at least, think that is why most Christians could care less about AI/climate change/nuclear holocaust/etc.
I don't worry as much about killer robots as AIs governments to spy on and control people. I believe an authoritarian country will create capabilities to create automated communications control and then export it or use it subversely in non-aligned countries. People and countries will have little defense against it because it can be difficult to detect and difficult defend against.