There is no point of AI dominance where a nation becomes immune to safety risks. For both the U.S. and China, every advance in model capability inherently increases national vulnerability to misuse, accidents, or attacks, linking the two concepts inextricably.

A strange dynamic exists where the tech leaders building AI are also the loudest voices warning of its potential to destroy humanity. This dual narrative of immense promise and existential threat serves to centralize their power, positioning them as the only ones who can both create and control this technology.

Small, seemingly harmless instances of reward hacking today are direct evidence for existential risk. There is no natural cutoff point where a slightly misaligned model will suddenly 'become good' once it gains world-altering capabilities.

AI offers incredible short-term benefits, from fixing daily problems to curing diseases. This immediate positive reinforcement makes it extremely difficult for society to acknowledge and address the simultaneous development of long-term, catastrophic risks, creating a classic devil's bargain.

The core risk of advanced AI is that its capabilities are dual-use. An AI superhuman at coding is also superhuman at hacking. An AI that designs cures can also design poisons. This inherent duality makes robust guardrails, which are largely absent in open-weight models, critical for safety.

The point of no return isn't AI as a powerful tool that enhances humans. It's when an autonomous AI, operating without oversight, can consistently outcompete a human in all relevant domains—from business to warfare. This shift from tool to autonomous competitor is the critical threshold for existential risk.

AI will create negative consequences, like the internet spawned the dark web. However, its potential to solve major problems like disease and energy scarcity makes its development a net positive for society, justifying the risks that must be managed along the way.

The current AI development model allows a small number of founders and firms to accumulate staggering wealth while the broader public bears the potential downside, from job displacement to existential threats. This mirrors past industrial shifts where private entities capitalized on innovation while externalizing the negative consequences.

The public discourse is dominated by 'P-Doom,' the probability of an AI-induced catastrophe. This focus obscures the other side of the equation: 'P-Boom,' the probability of unprecedented human flourishing. A balanced conversation requires acknowledging that if AI is powerful enough to destroy humanity, it is also powerful enough to radically improve it.