What would “AI killing us all” actually mean?

The phrase can hide several different claims. They should not be blended together.

At one end is ordinary harm caused with AI: fraud, harassment, deepfakes, cybercrime, discrimination, unsafe automation, misinformation, privacy loss, or a bad decision made by a system people trusted too much. These harms are already real in various forms, even if AI is only one factor in them.

A second category is catastrophic societal harm: events that could kill very large numbers of people, disable essential infrastructure, destabilize governments, trigger military escalation, or create global economic disruption. AI might contribute as a tool used by people, as an unreliable component inside a critical system, or as an autonomous agent given too much authority.

A third category is existential risk. In the strictest sense, this means outcomes that permanently destroy humanity’s long-term potential. Human extinction is the clearest example, but some definitions also include irreversible global subjugation or permanent loss of meaningful human control over civilization.

These are radically different levels of claim. A model helping a criminal write phishing emails does not prove that a superintelligent system will cause extinction. At the same time, demonstrating that current models are limited does not prove that future systems will remain harmless. The debate is difficult precisely because evidence exists at one level while fears often concern a much more advanced level.

A useful rule is: never let evidence about today silently become a conclusion about tomorrow, and never let a story about tomorrow masquerade as evidence about today.