An Anthropic alignment lead publicly stated a >10% chance of human extinction from AI within a decade. This creates a paradox: if a company truly believes its work carries such a high risk of global catastrophe, the logical response would be to shut down, not to continue development while trying to solve alignment.

The conversation about AI causing human extinction isn't led by outsiders but by insiders. After a researcher resigned from AI firm Anthropic over safety concerns, his former boss—the head of the AI safety team—publicly agreed, estimating the chance of AI ending the world at a staggering 10%.

Public and expert opinions on AI are split between two extremes: it will either save humanity or destroy it. There is a notable absence of a moderate, middle-ground perspective, which is a departure from how previous technological shifts like the internet were discussed.

Public proclamations of AI-driven extinction, like an Anthropic researcher's 10% odds of human annihilation, may be a deliberate strategy. By presenting worst-case scenarios, these individuals aim to trigger urgent conversations and push the industry and regulators toward implementing stronger safety measures.

Unlike past technological shifts, AI's ultimate impact is subject to violent disagreement among the world's top experts, including Nobel laureates. The spectrum of potential outcomes ranges from global utopia to human extinction, representing a historically unprecedented level of uncertainty that makes investment and planning exceptionally difficult.

While not a consensus, surveys of AI researchers reveal significant concern. The median respondent in a large survey assigned a 5% probability to human extinction or a similar disaster from AI, with a third to a half placing the risk at 10% or higher, suggesting the threat is taken seriously within the field.

The core disagreement between AI safety advocate Max Tegmark and former White House advisor Dean Ball stems from their vastly different probabilities of AI-induced doom. Tegmark’s >90% justifies preemptive regulation, while Ball’s 0.01% favors a reactive, innovation-friendly approach. Their policy stances are downstream of this fundamental risk assessment.

Sam Harris highlights the bizarre cultural phenomenon of AI leaders openly stating high probabilities (e.g., 20%) for existential risk while racing to build the technology. He contrasts this with Manhattan Project scientists, who proceeded only after calculating the risk of igniting the atmosphere as infinitesimal, not a double-digit percentage.

A thought experiment—pressing one of 1,000 buttons where 999 cure diseases and one ends humanity—exposes the core philosophical rift in the AI debate. One side takes the utilitarian bet for immense progress, while the other refuses to consent to any existential gamble on behalf of 8 billion people.