Coined in 1965, the "intelligence explosion" describes a runaway feedback loop. An AI capable of conducting AI research could use its intelligence to improve itself. This newly enhanced intelligence would make it even better at AI research, leading to exponential, uncontrollable growth in capability. This "fast takeoff" could leave humanity far behind in a very short period.

If society gets an early warning of an intelligence explosion, the primary strategy should be to redirect the nascent superintelligent AI 'labor' away from accelerating AI capabilities. Instead, this powerful new resource should be immediately tasked with solving the safety, alignment, and defense problems that it creates, such as patching vulnerabilities or designing biodefenses.

The property rights argument for AI safety hinges on an ecosystem of multiple, interdependent AIs. The strategy breaks down in a scenario where a single AI achieves a rapid, godlike intelligence explosion. Such an entity would be self-sufficient and could expropriate everyone else without consequence, as it wouldn't need to uphold the system.

AI leaders aren't ignoring risks because they're malicious, but because they are trapped in a high-stakes competitive race. This "code red" environment incentivizes patching safety issues case-by-case rather than fundamentally re-architecting AI systems to be safe by construction.

Ryan Kidd argues that it's nearly impossible to separate AI safety and capabilities work. Safety improvements, like RLHF, make models more useful and steerable, which in turn accelerates demand for more powerful "engines." This suggests that pure "safety-only" research is a practical impossibility.

The fundamental challenge of creating safe AGI is not about specific failure modes but about grappling with the immense power such a system will wield. The difficulty in truly imagining and 'feeling' this future power is a major obstacle for researchers and the public, hindering proactive safety measures. The core problem is simply 'the power.'

For any given failure mode, there is a point where further technical research stops being the primary solution. Risks become dominated by institutional or human factors, such as a company's deliberate choice not to prioritize safety. At this stage, policy and governance become more critical than algorithms.

The current approach to AI safety involves identifying and patching specific failure modes (e.g., hallucinations, deception) as they emerge. This "leak by leak" approach fails to address the fundamental system dynamics, allowing overall pressure and risk to build continuously, leading to increasingly severe and sophisticated failures.

The most likely reason AI companies will fail to implement their 'use AI for safety' plans is not that the technical problems are unsolvable. Rather, it's that intense competitive pressure will disincentivize them from redirecting significant compute resources away from capability acceleration toward safety, especially without robust, pre-agreed commitments.