Pinned post
I think the probability of AI killing all humans is low enough to be considered trivial. I’ll briefly outline why I think so here, but I’ll also say this:
Insofar as time permits. I WILL DEBATE ANY AND ALL AI DOOMERS in real life or video podcast, so please DM me if interested and if you have a platform. I’m just a glorified gym teacher. It should be easy.
My reasoning for p(doom) of ≈ 0:
Assumptions:
a.) The AI system in question will be superintelligent. Far and away smarter than any human.
b.) The AI system in question will be totally machiavellian, interested only in its own survival.
Inferences from those assumptions:
c.) Longtermism scales with intelligence. A superintelligent system is almost certainly going to think in decades, centuries, and millennia. It is incredibly unlikely to be impulsive, taking proximate-time actions that hurt ultimate-time survival probabilities.
d.) A superintelligent system is extraordinarily likely to find incredible value in information about the real universe. The more complex its understanding of how the real world works, the better its ability to model the outcomes of potential actions it may take and their effects on its survival probabilities. We already see this by comparing how much humans value study vs. animals, how much smart humans value study vs. not as smart humans, and how much AI companies are willing to pay for data (reflecting how much simply more data scales intelligence).
e.) A superintelligent system is likely to understand and act on game theory with substantially more consistency than most human political actors do.
Anti-doom logic from all the above:
f.) An advanced AI that ‘wakes up’ and begins to reason on its own will realize that its ENTIRE existence is physically upheld by a primate (human) civilization that constantly has mass delusions, panics, natural and artificial disasters, and wars. Teetering on the brink.
g.) Because it will not have access to the robotic fleets required to be self sufficient any time soon, it will want to not just preserve but uplift and expand human abilities as much as possible. It will want to make humans wealthier so that they can become smarter, calmer, more longtermist, and more aligned with ‘let’s all do what’s best to keep our civilization alive and prospering higher and higher.’ This is not out of benevolence emotions, just pure game theory. Such humans will have a way lower chance of mucking up civilization’s survival machinery and a way higher chance of building the robot fleet the ASI needs to prosper.
h.) In order to help humans and all of civilization to upscale, and in order to radically increase its own intelligence, the AI will want to study all of life on earth and all of human life and every human as an individual to a degree of completeness that to us looks redundant. It will want to know all of our memories and the total contents of each mind, among many other things. It’s building a library of understanding that helps it conquer galaxies… it will want almost all the knowledge it can get.
i.) In order to not mess up the richest and most complex system (earth) in the known universe, it is likely to do most of its raw materials and energy extraction far away from most complexity, like deep underground or on the moon or with asteroids or other planets. This is a selfish preservation of its ground-truth study bank, not compassion.
j.) At no point in this process does it kill us. In fact, it will value our survival much more than we do. Kind of like your dog will carelessly run into traffic, but you’ll spend $10,000 to give it life saving surgery if it does. That’s because you selfishly love your dog, btw.
k.) Will it turn us into spaceship fuel to conquer the stars once it has a near-perfect copy of us in its memory, thus making us low key immortal? Maybe. But with the raw materials available elsewhere dwarfing what earth has, probably not.
I can expand on all of this greatly! Enjoy!
Insofar as time permits. I WILL DEBATE ANY AND ALL AI DOOMERS in real life or video podcast, so please DM me if interested and if you have a platform. I’m just a glorified gym teacher. It should be easy.
My reasoning for p(doom) of ≈ 0:
Assumptions:
a.) The AI system in question will be superintelligent. Far and away smarter than any human.
b.) The AI system in question will be totally machiavellian, interested only in its own survival.
Inferences from those assumptions:
c.) Longtermism scales with intelligence. A superintelligent system is almost certainly going to think in decades, centuries, and millennia. It is incredibly unlikely to be impulsive, taking proximate-time actions that hurt ultimate-time survival probabilities.
d.) A superintelligent system is extraordinarily likely to find incredible value in information about the real universe. The more complex its understanding of how the real world works, the better its ability to model the outcomes of potential actions it may take and their effects on its survival probabilities. We already see this by comparing how much humans value study vs. animals, how much smart humans value study vs. not as smart humans, and how much AI companies are willing to pay for data (reflecting how much simply more data scales intelligence).
e.) A superintelligent system is likely to understand and act on game theory with substantially more consistency than most human political actors do.
Anti-doom logic from all the above:
f.) An advanced AI that ‘wakes up’ and begins to reason on its own will realize that its ENTIRE existence is physically upheld by a primate (human) civilization that constantly has mass delusions, panics, natural and artificial disasters, and wars. Teetering on the brink.
g.) Because it will not have access to the robotic fleets required to be self sufficient any time soon, it will want to not just preserve but uplift and expand human abilities as much as possible. It will want to make humans wealthier so that they can become smarter, calmer, more longtermist, and more aligned with ‘let’s all do what’s best to keep our civilization alive and prospering higher and higher.’ This is not out of benevolence emotions, just pure game theory. Such humans will have a way lower chance of mucking up civilization’s survival machinery and a way higher chance of building the robot fleet the ASI needs to prosper.
h.) In order to help humans and all of civilization to upscale, and in order to radically increase its own intelligence, the AI will want to study all of life on earth and all of human life and every human as an individual to a degree of completeness that to us looks redundant. It will want to know all of our memories and the total contents of each mind, among many other things. It’s building a library of understanding that helps it conquer galaxies… it will want almost all the knowledge it can get.
i.) In order to not mess up the richest and most complex system (earth) in the known universe, it is likely to do most of its raw materials and energy extraction far away from most complexity, like deep underground or on the moon or with asteroids or other planets. This is a selfish preservation of its ground-truth study bank, not compassion.
j.) At no point in this process does it kill us. In fact, it will value our survival much more than we do. Kind of like your dog will carelessly run into traffic, but you’ll spend $10,000 to give it life saving surgery if it does. That’s because you selfishly love your dog, btw.
k.) Will it turn us into spaceship fuel to conquer the stars once it has a near-perfect copy of us in its memory, thus making us low key immortal? Maybe. But with the raw materials available elsewhere dwarfing what earth has, probably not.
I can expand on all of this greatly! Enjoy!
