A key, informal safety layer against AI doom is the institutional self-preservation of the developers themselves. It's argued that labs like OpenAI or Google would not knowingly release a model they believed posed a genuine threat of overthrowing the government, opting instead to halt deployment and alert authorities.

Delaying public model releases isn't a real solution for pacing AI development. The critical risk lies in a lab using its most advanced, unreleased models to accelerate internal R&D, potentially leading to a private "singularity." Meaningful pacing must limit the resources dedicated to this internal recursive loop.

Leaders at top AI labs publicly state that the pace of AI development is reckless. However, they feel unable to slow down due to a classic game theory dilemma: if one lab pauses for safety, others will race ahead, leaving the cautious player behind.

Zvi Moshwitz claims OpenAI and Anthropic are "screaming" through public statements that their internal model capabilities are advancing at an unmanageable pace. These announcements are not just marketing but expressions of genuine fear that supervision, infrastructure, and safety measures cannot keep up with the accelerating progress they are witnessing.

Major AI companies are not solely seeking to stifle competition with regulation. They are also signaling an inability to self-regulate amidst intense competitive pressure, effectively asking external bodies to impose a mandatory safety floor that applies to everyone.

Regulatory focus on publicly released AI models overlooks the significant dangers from risky research and "internal deployment" within AI labs. True oversight requires visibility into these internal activities, not just the final products.

Investor David Sacks posits that OpenAI and Anthropic, as the dominant players, don't need external permission or regulation to slow their own development. He suggests their public calls for 'pacing' are a pretense to establish a regulatory cartel that would entrench their lead and police competitors under the guise of safety.

By publicly suggesting AI could wipe out humanity to justify their preferred regulations, lab leaders are likely to get far more draconian government intervention than they bargained for. The assumption they can control the regulatory outcome is viewed by critics as dangerously naive.

Calls to slow AI development aren't just regulatory capture. Didi Das notes that researchers at top labs are exposed to models far more advanced than the public sees, and many are "genuinely scared" by their capabilities, independent of financial incentives. This fear stems from direct, privileged access to future technology.