Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Unlike nuclear weapons, AI is too complex and fast-moving to be managed by formal treaties alone. True safety requires 'organic transparency,' where rich economic, cultural, and scientific engagement between nations builds the trust and informal oversight necessary to prevent catastrophe.

Related Insights

The US and USSR, despite being adversaries, collaborated to prevent nuclear proliferation to rogue actors. A similar model can be applied to AI. The US and China share an interest in preventing powerful open-weight models from being used for cyber-attacks or bio-terrorism by third parties, creating a foundation for a safety dialogue.

The path to surviving superintelligence is political: a global pact to halt its development, mirroring Cold War nuclear strategy. Success hinges on all leaders understanding that anyone building it ensures their own personal destruction, removing any incentive to cheat.

The same governments pushing AI competition for a strategic edge may be forced into cooperation. As AI democratizes access to catastrophic weapons (CBRN), the national security risk will become so great that even rival superpowers will have a mutual incentive to create verifiable safety treaties.

Counterintuitively, a multilateral AGI project led by a coalition of democracies is preferable to a single nation developing it in secret. A coalition creates checks and balances, as member countries would insist on safeguards to prevent the AGI from being used to install an authoritarian leader in any one nation.

Formal AI verification is difficult. The necessary trust can be built through "organic transparency"—the informal knowledge gained from deep economic, cultural, and scientific engagement. When business people and scientists from rival nations interact frequently, it creates a baseline understanding that makes formal governance agreements more achievable.

The belief that AI development is unstoppable ignores history. Global treaties successfully limited nuclear proliferation, phased out ozone-depleting CFCs, and banned blinding lasers. These precedents prove that coordinated international action can steer powerful technologies away from the worst outcomes.

The common analogy between regulating AI and nuclear weapons is flawed. Nuclear development requires physically trackable, interceptable materials and facilities like enrichment plants. In contrast, AI models are software and weights, which are diffuse and far more difficult to monitor and control, presenting a fundamentally different and harder regulatory challenge.

International AI treaties, particularly with nations like China, are unlikely to hold based on trust alone. A stable agreement requires a mutually-assured-destruction-style dynamic, meaning the U.S. must develop and signal credible offensive capabilities to deter cheating.

Nations don't need to like each other to cooperate on AI safety. The key is 'cognitive empathy'—the rational ability to understand another party's motivations and perspective. This is sufficient for navigating the non-zero-sum dynamics of global AI risk without requiring emotional warmth.

International AI treaties are feasible. Just as nuclear arms control monitors uranium and plutonium, AI governance can monitor the choke point for advanced AI: high-end compute chips from companies like NVIDIA. Tracking the global distribution of these chips could verify compliance with development limits.