Pinned post
AI takeover scenarios often hinge on an idea I’m skeptical of, where rogue AI overthrows militaries and seizes hard power. I wrote a piece where I concretely break down what I think makes this improbable, and why “full-blown takeover” concerns could be counterproductive for mitigating the risks of misaligned or poorly controlled AI in the military.
For AI to seize hard power, it would need to be physically capable of defeating our military, and humans would need to be careless enough to give it the power to do so. I think that’s unlikely to occur for several reasons. Among them:
- The military is unlikely to sufficiently hand the reins over to AI. (1) Even if we wanted to, overhauling the institution in this way faces countless barriers that have plagued it for decades. (2) Doing so would be strategically suicidal.
- Integrating general-purpose AI into physical systems faces challenges that will take significant time to overcome, all while we grow increasingly attuned to the risks posed by AI.
- There’s little to suggest that more capable AI would be able to persuade or coerce people to take the costly, coordinated actions some of these scenarios predict.
- Even if AI managed to assume control over a large arsenal, we’d fight back. In which case, recent warfare shows us that crude countermeasures work against technologically advanced adversaries, and AI relies on infrastructure we can and would shut down.
1/2
For AI to seize hard power, it would need to be physically capable of defeating our military, and humans would need to be careless enough to give it the power to do so. I think that’s unlikely to occur for several reasons. Among them:
- The military is unlikely to sufficiently hand the reins over to AI. (1) Even if we wanted to, overhauling the institution in this way faces countless barriers that have plagued it for decades. (2) Doing so would be strategically suicidal.
- Integrating general-purpose AI into physical systems faces challenges that will take significant time to overcome, all while we grow increasingly attuned to the risks posed by AI.
- There’s little to suggest that more capable AI would be able to persuade or coerce people to take the costly, coordinated actions some of these scenarios predict.
- Even if AI managed to assume control over a large arsenal, we’d fight back. In which case, recent warfare shows us that crude countermeasures work against technologically advanced adversaries, and AI relies on infrastructure we can and would shut down.
1/2