Even CEOs who fear AI's existential risks are rationally forced to accelerate development to keep up with competitors. This classic prisoner's dilemma makes voluntary slowdowns economically unstable, necessitating external regulation like 'circuit breakers.'

AI leaders don't slow down for safety because of a classic "tragedy of the commons." If one company pauses, competitors will race ahead to capture the market and IPO rewards. They publicly call for regulation as a way to force a collective, industry-wide slowdown they can't achieve on their own.

CEOs like Anthropic's Dario Amadei publicly advocate for slowing AI development and government oversight. The cynical but plausible take is this is a strategy to create a "regulatory capture" scenario, building a moat against open-source competition and ensuring their company's survival and market position.

Top AI companies like OpenAI and Anthropic cannot unilaterally slow development, even with safety concerns. They fear that competitors or foreign adversaries would seize an insurmountable advantage, forcing them to seek government-led coordination to pace development safely.

Leaders at top AI labs publicly state that the pace of AI development is reckless. However, they feel unable to slow down due to a classic game theory dilemma: if one lab pauses for safety, others will race ahead, leaving the cautious player behind.

The "Pacing the Frontier" letter, where AI employees ask for government-mandated slowdowns, highlights a prisoner's dilemma. No single lab can afford to slow down due to "competitive pressure" unless all are forced to do so simultaneously through regulation. This coordination problem is why they appeal to an external authority.

Regulating AI progress is a game-theoretic challenge, not a technical one. Like climate change, the rational choice for any single company or country is to defect from a slowdown agreement to gain a competitive edge. This pushes all actors toward a race that collectively increases risk and leads to the worst possible outcome.

Major AI companies are not solely seeking to stifle competition with regulation. They are also signaling an inability to self-regulate amidst intense competitive pressure, effectively asking external bodies to impose a mandatory safety floor that applies to everyone.

Governments face a difficult choice with AI regulation. Those that impose strict safety measures risk falling behind nations with a laissez-faire approach. This creates a global race condition where the fear of being outcompeted may discourage necessary safeguards, even when the risks are known.