We scan new podcasts and send you the top 5 insights daily.
Tom Milton says the fallback is turning data centers off, but that is extremely costly. Verification lets a prover show it is serving only inference, or running small safety experiments, so a slowdown can preserve useful work while stopping dangerous training. This reduces the cost of reaching a deal.
Implementing AI safety guardrails is not cost-prohibitive. The most impactful step, having a second AI model review the primary agent's work, is also the cheapest, accounting for only about 3% of total API costs in the author's experience. This makes it the most efficient first step for improving reliability.
Rather than trying to solve intractable philosophical alignment issues under intense pressure, governments can enforce a technical brake pedal by regulating compute allocation in data centers. Leading AI labs currently split their computing capacity roughly 50-50 between training next-generation models and serving inference to customers. Mandating that labs divert nearly all compute toward serving existing models effectively slows frontier escalation while preserving economic utility.
Formally verifying the vast weights of complex neural networks or multi-agent architectures is practically impossible. Instead, developers can use formal methods to construct provably correct external runtime monitors that constrain agents to safe operational boundaries and handle recovery when violations occur.
In an AI crisis, without trusted verification methods, the only US recourse with China would be an outlandish demand like shutting down large data centers. This high-cost, escalatory 'ask' is a direct result of not investing in nuanced verification and compliance tools beforehand.
Tom Milton argues verification is not only for outside regulators. If AI models gain nation-state-level cyber capability, labs need hardware-backed logs to confirm models have not changed their own weights or spoofed evaluation results. The same stack protects against untrusted AI as against an adversary country.
Any international or inter-company agreements to slow down AI development are impotent without the technology to verify compliance. The 'teeth' of any treaty or law will be technical verification tools, making their development a critical and urgent priority.
Tom Milton says he is not convinced verification is strictly necessary. Nuclear deals relied partly on trust, satellites counting bombers, and spying; AI deals could also use auditors, whistleblowers, or unilateral slowdowns. But as more lab leaders and governments call for coordination, verification becomes valuable optionality.
Instead of supervising an AI's hidden thought process, we can demand it produces a 'certificate of reasoning'—a checkable proof—along with its output. This could include citations or sensitivity analyses, shifting verification from observing the process to checking the provided proof.
After a security incident, OpenAI paused frontier model training to improve safety protocols. This self-regulation is a strategic move to build trust with enterprises and the public, suggesting that demonstrating safety will increasingly dictate the pace of AI progress and become a key business advantage.
Frontier AI labs struggle to honor voluntary training pause commitments because they fear cautious pauses will yield lead time to reckless competitors. Embedding independent auditors directly inside labs verifies genuine compliance across firms, making collective risk management and pause thresholds credible.