Coined in 1965, the "intelligence explosion" describes a runaway feedback loop. An AI capable of conducting AI research could use its intelligence to improve itself. This newly enhanced intelligence would make it even better at AI research, leading to exponential, uncontrollable growth in capability. This "fast takeoff" could leave humanity far behind in a very short period.

Fears of AI's 'recursive self-improvement' should be contextualized. Every major general-purpose technology, from iron to computers, has been used to improve itself. While AI's speed may differ, this self-catalyzing loop is a standard characteristic of transformative technologies and has not previously resulted in runaway existential threats.

The justification for accelerating AI development to beat China is logically flawed. It assumes the victor wields a controllable tool. In reality, both nations are racing to build the same uncontrollable AI, making the race itself, not the competitor, the primary existential threat.

The development of AI won't stop because of game theory. For competing nations like the US and China, the risk of falling behind is greater than the collective risk of developing the technology. This dynamic makes the AI race an unstoppable force, mirroring the Cold War nuclear arms race and rendering calls for a pause futile.

The first entity to achieve AGI could see it self-improve at an exponential rate, potentially achieving 20,000 years of progress overnight. This concept of "fast takeoff" makes any delay in the AI race, even for regulatory reasons, a potentially catastrophic strategic error.

Unlike any prior tool, AI can be directly applied to improve its own creation. It designs more efficient computer chips, writes better training code, and automates research, creating a recursive self-improvement loop that rapidly outpaces human oversight and control.

A cynical explanation for the race to build superintelligence is the immense power it would confer. The controller could develop technologies so advanced they would have a decisive advantage over all other global actors, akin to a group with guns facing one with swords.

Companies like OpenAI and Anthropic are not just building better models; their strategic goal is an "automated AI researcher." The ability for an AI to accelerate its own development is viewed as the key to getting so far ahead that no competitor can catch up.

The immense strategic advantage offered by AI ensures its development will continue, regardless of safety concerns from insiders. Much like the Manhattan Project, which proceeded despite catastrophic risk, the logic of "if we don't, China will" makes unilateral cessation of research impossible for any major power.