Martin Wolf frames AI not as just a technology but as a philosophical pact. We are gaining a powerful servant that raises existential questions about humanity's purpose and creates terrifying risks like unaccountable decision-making and AI-run armies.

The existential threat from AI isn't about controlling the technology, but about humanity controlling itself. The challenge is a 'God test' requiring a moral upgrade—overcoming our innate, self-serving cognitive biases to achieve the global cooperation needed to manage AI safely.

The long-held belief that direct human oversight can solve AI risks is breaking down. With sophisticated and dynamic systems, especially agentic ones, a human cannot meaningfully monitor operations in real-time. The solution is shifting towards automated, AI-driven governance and monitoring at higher levels of abstraction.

The point of no return isn't AI as a powerful tool that enhances humans. It's when an autonomous AI, operating without oversight, can consistently outcompete a human in all relevant domains—from business to warfare. This shift from tool to autonomous competitor is the critical threshold for existential risk.

The most immediate danger from AI is not a hypothetical superintelligence but the growing delta between AI's capabilities and the public's understanding of how it works. This knowledge gap allows for subtle, widespread behavioral manipulation, a more insidious threat than a single rogue AGI.

The true danger of AI is not a cinematic robot uprising, but a slow erosion of human agency. As we replace CEOs, military strategists, and other decision-makers with more efficient AIs, we gradually cede control to inscrutable systems we don't understand, rendering humanity powerless.

The fundamental challenge of creating safe AGI is not about specific failure modes but about grappling with the immense power such a system will wield. The difficulty in truly imagining and 'feeling' this future power is a major obstacle for researchers and the public, hindering proactive safety measures. The core problem is simply 'the power.'

Yoshua Bengio believes that as a technical solution to the AI control problem seems more plausible, the concentration of AI power in human hands to create a global dictatorship has become an even more likely catastrophic outcome. This shifts the primary x-risk from technical failure to malicious human use.

AI's real threat isn't Skynet, but its ability to accelerate society's 'metabolic rate' beyond human capacity for adaptation. This creates constant reorientation, instability, and ultimately a crisis of legitimacy in our institutions.