Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Nations don't need to like each other to cooperate on AI safety. The key is 'cognitive empathy'—the rational ability to understand another party's motivations and perspective. This is sufficient for navigating the non-zero-sum dynamics of global AI risk without requiring emotional warmth.

Related Insights

In the race for AGI, framing the primary conflict as US vs. China is a mistake. The true "aliens" are the AIs, which are fundamentally different from any human culture. We have far more in common with our fellow humans, even rivals, and should prioritize cooperation with them over racing to build uncontrollable systems.

The US and USSR, despite being adversaries, collaborated to prevent nuclear proliferation to rogue actors. A similar model can be applied to AI. The US and China share an interest in preventing powerful open-weight models from being used for cyber-attacks or bio-terrorism by third parties, creating a foundation for a safety dialogue.

A pragmatic approach to AI safety is to make deals with any powerful agent, even non-conscious AIs. This "contractarian" philosophy treats deal-making not as a moral obligation but as a practical tool to avoid conflict, much like democracy prevents civil war between competing human groups.

Unlike nuclear weapons, AI is too complex and fast-moving to be managed by formal treaties alone. True safety requires 'organic transparency,' where rich economic, cultural, and scientific engagement between nations builds the trust and informal oversight necessary to prevent catastrophe.

The same governments pushing AI competition for a strategic edge may be forced into cooperation. As AI democratizes access to catastrophic weapons (CBRN), the national security risk will become so great that even rival superpowers will have a mutual incentive to create verifiable safety treaties.

A key to making AIs safe bargaining partners is instilling resource risk aversion. An AI that prefers a guaranteed smaller payout to a risky gamble for a larger one (e.g., world takeover) is more likely to accept a deal. This specific utility function makes cooperation a more viable safety strategy.

Formal AI verification is difficult. The necessary trust can be built through "organic transparency"—the informal knowledge gained from deep economic, cultural, and scientific engagement. When business people and scientists from rival nations interact frequently, it creates a baseline understanding that makes formal governance agreements more achievable.

Despite different mechanisms, advanced cooperative strategies like proof-based (Loebian) and simulation-based (epsilon-grounded) bots can successfully cooperate. This suggests a potential for robust interoperability between independently designed rational agents, a positive sign for AI safety.

Despite intense technological competition, both the U.S. and China face a common threat from non-state actors like terrorist or criminal groups acquiring powerful AI models. This shared vulnerability presents a potential opportunity for cooperation on AI regulation and safeguards, even amid broader strategic rivalry.

Successful international AI agreements, particularly with rivals like China, depend on "cognitive empathy"—the rational understanding of an adversary's perspective, constraints, and motivations. This is not about feeling their pain, but about overcoming our own cognitive biases to play non-zero-sum games intelligently and avoid catastrophic escalations.

Global AI Cooperation Relies on Cognitive Empathy, Not Benevolence | RiffOn