Even CEOs who fear AI's existential risks are rationally forced to accelerate development to keep up with competitors. This classic prisoner's dilemma makes voluntary slowdowns economically unstable, necessitating external regulation like 'circuit breakers.'

AI leaders don't slow down for safety because of a classic "tragedy of the commons." If one company pauses, competitors will race ahead to capture the market and IPO rewards. They publicly call for regulation as a way to force a collective, industry-wide slowdown they can't achieve on their own.

Top AI companies like OpenAI and Anthropic cannot unilaterally slow development, even with safety concerns. They fear that competitors or foreign adversaries would seize an insurmountable advantage, forcing them to seek government-led coordination to pace development safely.

Researchers recognize that model release cadences should slow down to properly address safety risks, but individual labs face a prisoner's dilemma. With multi-trillion-dollar valuations and hundreds of billions committed to non-cancelable compute infrastructure, any single firm that voluntarily pauses while competitors like OpenAI, Meta, and xAI continue releasing models faces rapid obsolescence and commercial extinction.

Top AI labs like Anthropic publicly state that slowing down AI development would benefit society. However, they are caught in a strategic trap: a unilateral pause is unviable. Without a global agreement, any lab that pauses simply allows less cautious competitors to seize the lead, potentially making the ecosystem less safe.

Leaders at top AI labs publicly state that the pace of AI development is reckless. However, they feel unable to slow down due to a classic game theory dilemma: if one lab pauses for safety, others will race ahead, leaving the cautious player behind.

The "Pacing the Frontier" letter, where AI employees ask for government-mandated slowdowns, highlights a prisoner's dilemma. No single lab can afford to slow down due to "competitive pressure" unless all are forced to do so simultaneously through regulation. This coordination problem is why they appeal to an external authority.

Regulating AI progress is a game-theoretic challenge, not a technical one. Like climate change, the rational choice for any single company or country is to defect from a slowdown agreement to gain a competitive edge. This pushes all actors toward a race that collectively increases risk and leads to the worst possible outcome.

Despite safety concerns from their own employees, AI labs are trapped in a prisoner's dilemma. Any single company that pauses development risks bankruptcy, and any nation that does so risks falling behind competitors like China. This creates a race that can only be paced through a coordinated, international agreement.