Focusing solely on military-style AI power grabs is too narrow. Extreme power concentration is more likely to emerge from a messy interplay of three factors: active seizures of control, massive economic shifts from automation, and the erosion of society's ability to understand reality (epistemics).

While mitigating catastrophic AI risks is critical, the argument for safety can be used to justify placing powerful AI exclusively in the hands of a few actors. This centralization, intended to prevent misuse, simultaneously creates the monopolistic conditions for the Intelligence Curse to take hold.

The decision to restrict powerful but dangerous AI models like Claude Mythos to a select group of large corporations for safety reasons risks creating a massive centralization of power. This gives these entities an insurmountable technological advantage over smaller players and the public.

Despite their different philosophies, both Vitalik Buterin and Guillaume Verdon agree that the greatest immediate danger is the concentration of AI power. They argue that whether by a single AI or a dictatorial government, such centralization threatens human agency and is a risk that must be actively fought.

Contrary to the idea of AI for all, the most powerful models will likely be restricted to a few high-paying clients to prevent distillation and maximize revenue. This creates a future where competitive advantage is defined by exclusive AI access, potentially allowing large incumbents to crush smaller competitors.

While AI alignment gets attention, the risk of AI concentrating immense power in the hands of a few actors (corporations or states) is arguably more neglected. This could enable unprecedented surveillance or create a single company with the economic power of a nation, posing a distinct and severe threat.

While a fast AI takeoff accelerates some risks, slower, more gradual AI progress still enables dangerous power concentration. Scenarios like a head of state subverting government AIs for personal loyalty or gradual economic disenfranchisement do not depend on a single company achieving a sudden, massive capability lead.

While often proposed to manage safety, a centralized, government-led AGI project is highly dangerous from a power concentration perspective. It removes checks and balances by consolidating immense capability within a single entity, whether it's one country or one company collaborating with the government.

Meredith Whittaker argues the biggest AI threat is not a sci-fi apocalypse, but the consolidation of power. AI's core requirements—massive data, computing infrastructure, and distribution channels—are controlled by a handful of established tech giants, further entrenching their dominance.