We scan new podcasts and send you the top 5 insights daily.
Tom Milton says he is not convinced verification is strictly necessary. Nuclear deals relied partly on trust, satellites counting bombers, and spying; AI deals could also use auditors, whistleblowers, or unilateral slowdowns. But as more lab leaders and governments call for coordination, verification becomes valuable optionality.
As sovereign AI initiatives grow, the risk of an unchecked capabilities race increases. A shared language of evaluations could provide a "trust but verify" mechanism, akin to nuclear arms verification treaties. This allows nations to audit each other's AI for risks like runaway RSI, creating a basis for international policy.
Unlike nuclear weapons, AI is too complex and fast-moving to be managed by formal treaties alone. True safety requires 'organic transparency,' where rich economic, cultural, and scientific engagement between nations builds the trust and informal oversight necessary to prevent catastrophe.
Tom Milton argues verification is not only for outside regulators. If AI models gain nation-state-level cyber capability, labs need hardware-backed logs to confirm models have not changed their own weights or spoofed evaluation results. The same stack protects against untrusted AI as against an adversary country.
Khosla asserts that international AI safety treaties are naive and impossible. Unlike nuclear or biotech, where usage can be verified, AI's deployment is covert. He argues that authoritarian states like China and Russia cannot be trusted to abide by agreements you cannot monitor, making such treaties ineffective and dangerous for the West.
Tom Milton says the fallback is turning data centers off, but that is extremely costly. Verification lets a prover show it is serving only inference, or running small safety experiments, so a slowdown can preserve useful work while stopping dangerous training. This reduces the cost of reaching a deal.
Formal AI verification is difficult. The necessary trust can be built through "organic transparency"—the informal knowledge gained from deep economic, cultural, and scientific engagement. When business people and scientists from rival nations interact frequently, it creates a baseline understanding that makes formal governance agreements more achievable.
Any international or inter-company agreements to slow down AI development are impotent without the technology to verify compliance. The 'teeth' of any treaty or law will be technical verification tools, making their development a critical and urgent priority.
For AI safety, Demis Hassabis advocates for an international regulatory body, similar to the International Atomic Energy Agency. This body would have technical experts who audit frontier models against agreed-upon benchmarks, checking for undesirable properties like deception and ensuring public confidence through independent verification.
Claims that AI treaties are unverifiable lack imagination. During the Cold War, the US and USSR agreed to saw bombers in half on runways, allowing spy planes to visually confirm disarmament. Similar 'outside-the-box' physical verification methods, like publicly escrowing or destroying GPUs, could work for AI.
International AI treaties are feasible. Just as nuclear arms control monitors uranium and plutonium, AI governance can monitor the choke point for advanced AI: high-end compute chips from companies like NVIDIA. Tracking the global distribution of these chips could verify compliance with development limits.