Contrary to popular cynicism, ominous warnings about AI from leaders like Anthropic's CEO are often genuine. Ethan Mollick suggests these executives truly believe in the potential dangers of the technology they are creating, and it's not solely a marketing tactic to inflate its power.

Public proclamations of AI-driven extinction, like an Anthropic researcher's 10% odds of human annihilation, may be a deliberate strategy. By presenting worst-case scenarios, these individuals aim to trigger urgent conversations and push the industry and regulators toward implementing stronger safety measures.

While not a consensus, surveys of AI researchers reveal significant concern. The median respondent in a large survey assigned a 5% probability to human extinction or a similar disaster from AI, with a third to a half placing the risk at 10% or higher, suggesting the threat is taken seriously within the field.

The most dire predictions of mass unemployment from AI come directly from its creators, like OpenAI's Sam Altman and xAI's Elon Musk. This contradicts the narrative that fear is driven by outsiders, suggesting those closest to the tech see its disruptive power most clearly.

Many top AI CEOs openly admit the extinction-level risks of their work, with some estimating a 25% chance. However, they feel powerless to stop the race. If a CEO paused for safety, investors would simply replace them with someone willing to push forward, creating a systemic trap where everyone sees the danger but no one can afford to hit the brakes.

The leaders of top AI labs have signed statements acknowledging AI could cause human extinction. Yet, a safety report gives them failing grades on 'existential safety,' finding it jarring that these same leaders are actively building superintelligence without any articulated plan for how to maintain human control over the technology.

The risk of human extinction from AI isn't just science fiction but a logical conclusion. If we successfully create an intelligence that is both smarter than us and capable of pursuing its own goals, there is no logical reason to believe we could maintain control over it.

Jacob Coxon's resignation from Anthropic went viral, forcing CEOs like Sam Altman and Dario Amadei to publicly agree on pacing AI development. This moment reflects a societal tipping point where the perceived risks of AI finally outweighed the race for capabilities, achieving what more famous experts previously could not.

Sam Harris highlights the bizarre cultural phenomenon of AI leaders openly stating high probabilities (e.g., 20%) for existential risk while racing to build the technology. He contrasts this with Manhattan Project scientists, who proceeded only after calculating the risk of igniting the atmosphere as infinitesimal, not a double-digit percentage.