The most pressing danger from AI isn't a hypothetical superintelligence but its use as a tool for societal control. The immediate risk is an Orwellian future where AI censors information, rewrites history for political agendas, and enables mass surveillance—a threat far more tangible than science fiction scenarios.

For some policy experts, the most realistic nightmare scenario is not a rogue superintelligence but a socio-economic collapse into techno-feudalism. In this future, AI concentrates power and wealth, creating a rentier state with a small ruling class and a large population with minimal economic agency or purpose.

The narrative that AI could be catastrophic ('summoning the demon') is used strategically. It creates a sense of danger that justifies why a small, elite group must maintain tight control over the technology, thereby warding off both regulation and competition.

Contrary to the idea of AI for all, the most powerful models will likely be restricted to a few high-paying clients to prevent distillation and maximize revenue. This creates a future where competitive advantage is defined by exclusive AI access, potentially allowing large incumbents to crush smaller competitors.

The public focus on hypothetical extinction scenarios overshadows immediate, tangible AI risks. These include sophisticated cybersecurity attacks, financial infrastructure vulnerabilities, and data privacy issues, such as OpenAI admitting user data could be used to train models on sensitive problems.

The AI extinction narrative strategically reinforces the idea that only expensive, proprietary frontier models are truly powerful. This counters the business threat from cheaper, open-source alternatives by re-centering the conversation on unique, high-stakes capabilities that only a few labs supposedly possess.

The true danger of AI is not a cinematic robot uprising, but a slow erosion of human agency. As we replace CEOs, military strategists, and other decision-makers with more efficient AIs, we gradually cede control to inscrutable systems we don't understand, rendering humanity powerless.

If AI makes intelligence cheap and universally available, its economic value may collapse. This theory suggests that selling raw AI models could become a low-margin, utility-like business. Profitability will depend on building moats through specialized applications or regulatory capture, not on selling base intelligence.

Meredith Whittaker argues the biggest AI threat is not a sci-fi apocalypse, but the consolidation of power. AI's core requirements—massive data, computing infrastructure, and distribution channels—are controlled by a handful of established tech giants, further entrenching their dominance.