If AGI is concentrated in a few US companies, other nations could lose their economic sovereignty. When American AGI can produce goods far cheaper than local human labor, economies like the UK's could collapse. They would become economically dependent "client states," reliant on American technology for almost all production, with wealth accruing to Silicon Valley.

As AI makes it increasingly easy to get answers without effort, society may split into two groups. Bernd Hobart suggests a "cognitive underclass" will opt for the ease of AI-generated solutions, while a "cognitive overclass" will deliberately engage in the now-optional hard work of critical thinking, creating a new societal divide.

While mitigating catastrophic AI risks is critical, the argument for safety can be used to justify placing powerful AI exclusively in the hands of a few actors. This centralization, intended to prevent misuse, simultaneously creates the monopolistic conditions for the Intelligence Curse to take hold.

AI provides a structural advantage to those in power by automating government systems. This allows leaders to bypass the traditional unwieldiness of human bureaucracy, making it trivial for an executive to change AI parameters and instantly exert their will across all levels of government, thereby concentrating power.

For some policy experts, the most realistic nightmare scenario is not a rogue superintelligence but a socio-economic collapse into techno-feudalism. In this future, AI concentrates power and wealth, creating a rentier state with a small ruling class and a large population with minimal economic agency or purpose.

For current AI valuations to be realized, AI must deliver unprecedented efficiency, likely causing mass job displacement. This would disrupt the consumer economy that supports these companies, creating a fundamental contradiction where the condition for success undermines the system itself.

AI experts like Eric Schmidt and Henry Kissinger predict AI will split society into two tiers: a small elite who develops AI and a large class that becomes dependent on it for decisions. This reliance will lead to "cognitive diminishment," where critical thinking skills atrophy, much like losing mental math abilities by overusing a calculator.

The most dangerous long-term impact of AI is not economic unemployment, but the stripping away of human meaning and purpose. As AI masters every valuable skill, it will disrupt the core human algorithm of contributing to the group, leading to a collective psychological crisis and societal decay.

As AIs increasingly perform all economically necessary work, the incentive for entities like governments and corporations to invest in human capital may disappear. This creates a long-term risk of a society where humans are no longer seen as a necessary resource to cultivate, leading to a permanent dependency.