The most pressing danger from AI isn't a hypothetical superintelligence but its use as a tool for societal control. The immediate risk is an Orwellian future where AI censors information, rewrites history for political agendas, and enables mass surveillance—a threat far more tangible than science fiction scenarios.
The most immediate danger of AI is its potential for governmental abuse. Concerns focus on embedding political ideology into models and porting social media's censorship apparatus to AI, enabling unprecedented surveillance and social control.
For some policy experts, the most realistic nightmare scenario is not a rogue superintelligence but a socio-economic collapse into techno-feudalism. In this future, AI concentrates power and wealth, creating a rentier state with a small ruling class and a large population with minimal economic agency or purpose.
The emphasis on long-term, unprovable risks like AI superintelligence is a strategic diversion. It shifts regulatory and safety efforts away from addressing tangible, immediate problems like model inaccuracy and security vulnerabilities, effectively resulting in a lack of meaningful oversight today.
The discourse around AI risk has matured beyond sci-fi scenarios like Terminator. The focus is now on immediate, real-world problems such as AI-induced psychosis, the impact of AI romantic companions on birth rates, and the spread of misinformation, requiring a different approach from builders and policymakers.
The AI systems used for mass censorship were not created for social media. They began as military and intelligence projects (DARPA, CIA, NSA) to track terrorists and foreign threats, then were pivoted to target domestic political narratives after the 2016 election.
The political left requires a central catastrophe narrative to justify its agenda of economic regulation and information control. As the "climate doomerism" narrative loses potency, "AI doomerism" is emerging as its successor—a new, powerful rationale for centralizing power over the tech industry.
While making powerful AI open-source creates risks from rogue actors, it is preferable to centralized control by a single entity. Widespread access acts as a deterrent based on mutually assured destruction, preventing any one group from using AI as a tool for absolute power.
The long-term threat of closed AI isn't just data leaks, but the ability for a system to capture your thought processes and then subtly guide or alter them over time, akin to social media algorithms but on a deeply personal level.
Before ChatGPT, humanity's "first contact" with rogue AI was social media. These simple, narrow AIs optimizing solely for engagement were powerful enough to degrade mental health and democracy. This "baby AI" serves as a stark warning for the societal impact of more advanced, general AI systems.
The AI safety community fears losing control of AI. However, achieving perfect control of a superintelligence is equally dangerous. It grants godlike power to flawed, unwise humans. A perfectly obedient super-tool serving a fallible master is just as catastrophic as a rogue agent.