Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Despite intense geopolitical rivalry in the AI race, the failure of the U.S. and China to cooperate on AI safety poses a catastrophic risk. Experts warn that without joint guardrails and arms control for AI-powered weapons, unchecked competition could lead to dystopian outcomes for civilization.

Related Insights

The ultimate goal for US-China relations should not be to "win" a zero-sum AI race, but to achieve a "Pax Robotica"—a future of shared peace and abundance. This reframes the relationship from competition to collaboration, avoiding the creation of new AI-powered existential risks.

Robert Wright argues the most immediate existential danger isn't a superintelligent AI takeover, but the geopolitical competition. A breakneck race for AI supremacy could provoke one nation to launch a preemptive war to prevent the other from achieving a perceived decisive advantage.

The US and USSR, despite being adversaries, collaborated to prevent nuclear proliferation to rogue actors. A similar model can be applied to AI. The US and China share an interest in preventing powerful open-weight models from being used for cyber-attacks or bio-terrorism by third parties, creating a foundation for a safety dialogue.

The same governments pushing AI competition for a strategic edge may be forced into cooperation. As AI democratizes access to catastrophic weapons (CBRN), the national security risk will become so great that even rival superpowers will have a mutual incentive to create verifiable safety treaties.

The greatest risk in US-China tech competition is not one side getting ahead, but their technology stacks becoming mutually unintelligible. This lack of legibility creates a volatile environment ripe for catastrophic miscalculation and conflict, making a shared AI paradigm a strategic necessity for peace.

Despite intense technological competition, both the U.S. and China face a common threat from non-state actors like terrorist or criminal groups acquiring powerful AI models. This shared vulnerability presents a potential opportunity for cooperation on AI regulation and safeguards, even amid broader strategic rivalry.

The AI competition is not a race to develop the most powerful technology, but a race to see which nation is better at steering and governing that power. Developing an uncontrollable 'AI bazooka' first is not a win; true advantage comes from creating systems that strengthen, rather than weaken, one's own society.

Global consensus on AI safety is unlikely soon. The most practical approach is a bilateral agreement between the US and China, the two dominant players. This can begin with informal 'track two' conversations between their respective AI labs before escalating to a formal government pact.

The intense U.S.-China rivalry prevents cooperation on AI safety, creating a dangerous vacuum. Without guardrails or arms control agreements, the risk of rogue AI agents or AI-powered weapons in nuclear or biological domains threatens to 'set back civilization.'

Unlike the Cold War's 'mutually assured destruction,' if one nation secretly believes it can control a superintelligence, it will pursue it. The only path to a stable pause is if both the US and China agree that superintelligence is uncontrollable by anyone, leading to 'assured destruction' for all, thus aligning their incentives to truly halt development.

U.S. and China Must Cooperate on AI Safety to Avoid Catastrophe, Despite Rivalry | RiffOn