While mitigating catastrophic AI risks is critical, the argument for safety can be used to justify placing powerful AI exclusively in the hands of a few actors. This centralization, intended to prevent misuse, simultaneously creates the monopolistic conditions for the Intelligence Curse to take hold.

Contrary to the view that AI competition is a 'dangerous race,' it is a positive force that protects consumers and fosters decentralization. This competition is the best defense against regulatory capture that could lead to a single, centralized AI becoming a totalitarian power.

The idea of nations collectively creating policies to slow AI development for safety is naive. Game theory dictates that the immense competitive advantage of achieving AGI first will drive nations and companies to race ahead, making any global regulatory agreement effectively unenforceable.

The intense competition and personal rivalries among AI lab leaders, while seemingly petty, serve as a structural safeguard. This prevents the formation of a monopoly on frontier AI. The resulting diversity in model weights and ownership makes a unilateral takeover by a single entity's AI far less likely than in a world with a unified development effort.

Instead of a single, all-powerful AGI emerging, the reality of AI is a "polytheistic" ecosystem of many decentralized models, each with different strengths. This framework challenges the notion of a single entity to control or fear and suggests a more complex, competitive landscape.

Despite their different philosophies, both Vitalik Buterin and Guillaume Verdon agree that the greatest immediate danger is the concentration of AI power. They argue that whether by a single AI or a dictatorial government, such centralization threatens human agency and is a risk that must be actively fought.

The AI competition is not a race to develop the most powerful technology, but a race to see which nation is better at steering and governing that power. Developing an uncontrollable 'AI bazooka' first is not a win; true advantage comes from creating systems that strengthen, rather than weaken, one's own society.

While AI alignment gets attention, the risk of AI concentrating immense power in the hands of a few actors (corporations or states) is arguably more neglected. This could enable unprecedented surveillance or create a single company with the economic power of a nation, posing a distinct and severe threat.

While often proposed to manage safety, a centralized, government-led AGI project is highly dangerous from a power concentration perspective. It removes checks and balances by consolidating immense capability within a single entity, whether it's one country or one company collaborating with the government.