Contrary to popular cynicism, ominous warnings about AI from leaders like Anthropic's CEO are often genuine. Ethan Mollick suggests these executives truly believe in the potential dangers of the technology they are creating, and it's not solely a marketing tactic to inflate its power.

The argument for rapidly advancing powerful AI is that only the leading labs can influence safety protocols. This 'stay in the lead to steer' philosophy creates a paradox: to mitigate AI risk, companies feel compelled to accelerate its development, potentially amplifying the very dangers they aim to control.

The narrative that AI could be catastrophic ('summoning the demon') is used strategically. It creates a sense of danger that justifies why a small, elite group must maintain tight control over the technology, thereby warding off both regulation and competition.

The most dire predictions of mass unemployment from AI come directly from its creators, like OpenAI's Sam Altman and xAI's Elon Musk. This contradicts the narrative that fear is driven by outsiders, suggesting those closest to the tech see its disruptive power most clearly.

A strange dynamic exists where the tech leaders building AI are also the loudest voices warning of its potential to destroy humanity. This dual narrative of immense promise and existential threat serves to centralize their power, positioning them as the only ones who can both create and control this technology.

Many top AI CEOs openly admit the extinction-level risks of their work, with some estimating a 25% chance. However, they feel powerless to stop the race. If a CEO paused for safety, investors would simply replace them with someone willing to push forward, creating a systemic trap where everyone sees the danger but no one can afford to hit the brakes.

Claims that AI CEOs use extinction risk as a marketing ploy are unconvincing. Many expressed these concerns long before leading major companies. Furthermore, highlighting catastrophic risk is a poor strategy for attracting investment and actively invites unwanted regulatory attention.

Leaders at top AI labs publicly state that the pace of AI development is reckless. However, they feel unable to slow down due to a classic game theory dilemma: if one lab pauses for safety, others will race ahead, leaving the cautious player behind.

Sam Harris highlights the bizarre cultural phenomenon of AI leaders openly stating high probabilities (e.g., 20%) for existential risk while racing to build the technology. He contrasts this with Manhattan Project scientists, who proceeded only after calculating the risk of igniting the atmosphere as infinitesimal, not a double-digit percentage.