Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Unlike the Cold War's 'mutually assured destruction,' if one nation secretly believes it can control a superintelligence, it will pursue it. The only path to a stable pause is if both the US and China agree that superintelligence is uncontrollable by anyone, leading to 'assured destruction' for all, thus aligning their incentives to truly halt development.

Related Insights

The justification for accelerating AI development to beat China is logically flawed. It assumes the victor wields a controllable tool. In reality, both nations are racing to build the same uncontrollable AI, making the race itself, not the competitor, the primary existential threat.

The path to surviving superintelligence is political: a global pact to halt its development, mirroring Cold War nuclear strategy. Success hinges on all leaders understanding that anyone building it ensures their own personal destruction, removing any incentive to cheat.

The development of AI won't stop because of game theory. For competing nations like the US and China, the risk of falling behind is greater than the collective risk of developing the technology. This dynamic makes the AI race an unstoppable force, mirroring the Cold War nuclear arms race and rendering calls for a pause futile.

The nuclear arms race precedent suggests China will inevitably develop powerful AI. The crucial policy question is not how to block their progress, but how to manage a world where they have achieved AI parity, a concept akin to mutually assured destruction that is currently missing from the US discourse.

Pausing or regulating AI development domestically is futile. Because AI offers a winner-take-all advantage, competing nations like China will inevitably lie about slowing down while developing it in secret. Unilateral restraint is therefore a form of self-sabotage.

The argument that the US must race China to AGI is flawed. If one country is close to an AGI that could neutralize the other's nuclear arsenal, it creates a powerful incentive for the losing side to launch a preemptive nuclear strike. This makes a mutual moratorium on superintelligence the more rational geopolitical strategy.

The immense strategic advantage offered by AI ensures its development will continue, regardless of safety concerns from insiders. Much like the Manhattan Project, which proceeded despite catastrophic risk, the logic of "if we don't, China will" makes unilateral cessation of research impossible for any major power.

Paradoxically, progress in AI alignment has made a global slowdown agreement impossible. Each side now trusts its own 'aligned' AI more than it trusts the other nation to uphold a deal. The perceived risk of being surpassed by a defector now outweighs the perceived risk of one's own AI going rogue, making a competitive race the rational choice.

A common argument against an AI development pause is that China wouldn't agree. However, since China is currently behind in the AI race, a pause is strategically beneficial for them as it stops the leader from extending their lead. This counterintuitive point suggests a pause is more geopolitically feasible than often assumed.

The race for AI supremacy is governed by game theory. Any technology promising an advantage will be developed. If one nation slows down for safety, a rival will speed up to gain strategic dominance. Therefore, focusing on guardrails without sacrificing speed is the only viable path.