Top Chinese officials use the metaphor "if the braking system isn't under control, you can't really step on the accelerator with confidence." This reflects a core belief that robust safety measures enable, rather than hinder, the aggressive development and deployment of powerful AI systems, viewing the two as synergistic.

If society gets an early warning of an intelligence explosion, the primary strategy should be to redirect the nascent superintelligent AI 'labor' away from accelerating AI capabilities. Instead, this powerful new resource should be immediately tasked with solving the safety, alignment, and defense problems that it creates, such as patching vulnerabilities or designing biodefenses.

Impactful AI for societal decision-making can be categorized into two main types. Epistemic tools help us understand what is true (e.g., AI fact-checkers, forecasters), while coordination tools help groups cooperate (e.g., AI negotiators, verification systems). This provides a clear framework for targeted development.

Framing an AI development pause as a binary on/off switch is unproductive. A better model is to see it as a redirection of AI labor along a spectrum. Instead of 100% of AI effort going to capability gains, a 'pause' means shifting that effort towards defensive activities like alignment, biodefense, and policy coordination, while potentially still making some capability progress.

AI accelerationists and safety advocates often appear to have opposing goals, but may actually desire a similar 10-20 year transition period. The conflict arises because accelerationists believe the default timeline is 50-100 years and want to speed it up, while safety advocates believe the default is an explosive 1-5 years and want to slow it down.

The risk of malicious actors using powerful AI decision tools is significant. The most effective countermeasure is not to restrict the technology, but to ensure it is widely and equitably distributed. This prevents any single group from gaining a dangerous strategic advantage over others.

Ryan Kidd argues that it's nearly impossible to separate AI safety and capabilities work. Safety improvements, like RLHF, make models more useful and steerable, which in turn accelerates demand for more powerful "engines." This suggests that pure "safety-only" research is a practical impossibility.

A key failure mode for using AI to solve AI safety is an 'unlucky' development path where models become superhuman at accelerating AI R&D before becoming proficient at safety research or other defensive tasks. This could create a period where we know an intelligence explosion is imminent but are powerless to use the precursor AIs to prepare for it.

Even if the market would eventually build decision-making tools, their impact is time-sensitive. Waiting for commercial rollout might mean they arrive after AGI, too late to help navigate the riskiest period. Therefore, philanthropic or impact-driven acceleration, even by a few months, is highly valuable.