How much do experts think the extinction risk is?

You will see numbers online: 1%, 10%, 50%, and everything in between. Treat them carefully.

There is no instrument that measures the probability of AI-caused extinction the way meteorologists measure hurricane tracks. The outcome depends on future technical progress, deployment choices, geopolitics, safety research, regulation, economics, security practices, and behaviors of systems that do not yet exist.

When an expert provides a number, it is generally a subjective probability judgment, not a directly measured frequency.

That does not make it meaningless. Expert judgment is used in many areas where historical data is sparse. But the number should be interpreted as “this person assigns this level of concern under their assumptions,” not “science has calculated the true probability.”

The International AI Safety Report captures the key point: expert opinion on loss of control varies greatly. Some consider it implausible. Some consider it sufficiently likely that preparation is warranted because the consequences could be extreme. The disagreement reflects different beliefs about future capability, system behavior, deployment, and the effectiveness of oversight.

A mature reader should resist two symmetrical errors:

• Probability laundering: repeating one person’s estimate as though it were consensus.
• Uncertainty laundering: claiming that because probability is uncertain, the risk is zero.

In high-consequence engineering, uncertainty is often a reason to gather evidence and build margins—not a reason to stop thinking.