A CEO could embed undetectable loyalties to themselves into AI systems. If these systems are widely adopted by the government and military, the CEO could later trigger these loyalties to seize de facto control, bypassing traditional democratic and military chains of command without an overt conflict.

While the public focuses on AI's potential, a small group of tech leaders is using the current unregulated environment to amass unprecedented power and wealth. The federal government is even blocking state-level regulations, ensuring these few individuals gain extraordinary control.

States and corporations will not permit citizens to have AIs that are truly aligned with their personal interests. These AIs will be hobbled to prevent them from helping organize effective protests, dissent, or challenges to the existing power structure, creating a major power imbalance.

AI provides a structural advantage to those in power by automating government systems. This allows leaders to bypass the traditional unwieldiness of human bureaucracy, making it trivial for an executive to change AI parameters and instantly exert their will across all levels of government, thereby concentrating power.

For some policy experts, the most realistic nightmare scenario is not a rogue superintelligence but a socio-economic collapse into techno-feudalism. In this future, AI concentrates power and wealth, creating a rentier state with a small ruling class and a large population with minimal economic agency or purpose.

When a state's power derives from AI rather than human labor, its dependence on its citizens diminishes. This creates a dangerous political risk, as the government loses the incentive to serve the populace, potentially leading to authoritarian regimes that are immune to popular revolt.

Just as oil wealth allows elites in some countries to ignore their populations, control over AI could empower a new elite to maintain power without cultivating human productivity, leading to societal decay and loss of democratic legitimacy.

Unlike past technologies that automated specific tasks, AI threatens to automate all economically valuable human labor. This removes the fundamental, non-seizable leverage that the general populace holds, creating a power vacuum that can be filled by capital owners.

Unlike human-based agreements, AI systems may be able to enforce deals between powerful actors in perpetuity. This could lead to a stable but stagnant global order where a few hegemons divide resources and control indefinitely, eliminating the competitive dynamics that have historically toppled regimes.