Framing AI governance as a short, time-limited 'pause' is a mistake. A better approach is a moratorium that ends not after a set time, but only when society's investment in safety and governance is 'commensurate' with the monumental risk of creating superintelligence, a standard we are currently failing to meet.

The path to surviving superintelligence is political: a global pact to halt its development, mirroring Cold War nuclear strategy. Success hinges on all leaders understanding that anyone building it ensures their own personal destruction, removing any incentive to cheat.

Vitalik Buterin suggests that slowing AI progress to buy time for safety is a valid goal. He argues the most feasible and least dystopian method is to limit hardware production. Since chip manufacturing is already highly centralized, it presents a control point that avoids more invasive, freedom-restricting measures.

Top AI labs like Anthropic publicly state that slowing down AI development would benefit society. However, they are caught in a strategic trap: a unilateral pause is unviable. Without a global agreement, any lab that pauses simply allows less cautious competitors to seize the lead, potentially making the ecosystem less safe.

A pause on training new, more capable AI models could paradoxically increase risk. It would halt progress at the few, relatively safety-conscious frontier labs, allowing less scrupulous competitors to catch up. Meanwhile, compute stockpiling would continue, making any subsequent capability leap even faster and more dangerous.

Framing an AI development pause as a binary on/off switch is unproductive. A better model is to see it as a redirection of AI labor along a spectrum. Instead of 100% of AI effort going to capability gains, a 'pause' means shifting that effort towards defensive activities like alignment, biodefense, and policy coordination, while potentially still making some capability progress.

Critics argue that proposals like 'AI 2040,' which advocate for government action to slow AI development, are dangerous. They propose granting governments unprecedented surveillance and control to fight a fictional threat, creating a tangible risk of authoritarian abuse that is worse than the problem it aims to solve.

The default assumption is that slowing innovation is inherently bad. With a technology as potent as AI, a deliberate slowdown is a feature, providing critical time to understand the systems, manage disruptions, and build governance structures before irreversible consequences occur. A true halt is not the alternative.

To truly stop "rogue AI," one would need to monitor every chip on the planet and use violence to stop unapproved computations. This path, advocated by some AI safety proponents, backs into a call for a totalitarian regime to put the technology "back in the box."