Ajeya Cotra reports that leading developers like OpenAI, Anthropic, and DeepMind are converging on a strategy where each generation of AI is used to help align, control, and understand the subsequent, more powerful generation. This recursive approach is their primary plan for ensuring AI safety during rapid takeoff.

Fears of AI's 'recursive self-improvement' should be contextualized. Every major general-purpose technology, from iron to computers, has been used to improve itself. While AI's speed may differ, this self-catalyzing loop is a standard characteristic of transformative technologies and has not previously resulted in runaway existential threats.

The path to surviving superintelligence is political: a global pact to halt its development, mirroring Cold War nuclear strategy. Success hinges on all leaders understanding that anyone building it ensures their own personal destruction, removing any incentive to cheat.

If society gets an early warning of an intelligence explosion, the primary strategy should be to redirect the nascent superintelligent AI 'labor' away from accelerating AI capabilities. Instead, this powerful new resource should be immediately tasked with solving the safety, alignment, and defense problems that it creates, such as patching vulnerabilities or designing biodefenses.

A ban on superintelligence is self-defeating because enforcement would require a sanctioned, global government body to build the very technology it prohibits in order to "prove it's safe." This paradoxically creates a state-controlled monopoly on the most powerful technology ever conceived, posing a greater risk than a competitive landscape.

One of the most promising and neglected AI safety strategies is to create systems for making credible deals with AIs. Just as contracts prevent conflict in human society, offering AIs guaranteed resources in exchange for cooperation makes rebellion a less attractive option.

The mismatch between exponentially advancing AI and slow, "medieval" institutions is a core risk. Instead of only focusing on recursively self-improving AI, we should apply technology to create self-improving governance systems that can adapt and update at the same speed as the challenges they face.

With no single silver bullet for AI alignment, the most realistic approach is a multi-layered strategy. This combines technical solutions like intentional design and AI control with societal safeguards like improved cybersecurity and pandemic preparedness to collectively keep society on track amidst rapid AI advancement.

The core safety challenge is that we have little understanding of how advanced AI systems function internally. We are essentially "growing" them through training, not engineering them with comprehensible parts. This means we cannot verify their true goals, making safety measures a gamble on observed behavior.