Aza Raskin argues the danger of AI is not the technology itself, but the economic incentives driving it. The debate is framed as a competition for resources between AI development and human needs, creating a future where technological progress comes at the direct expense of humanity.

An initial era of AI-driven superabundance will eventually end as the machine economy hits new resource limits (e.g., land, energy). At this point, the opportunity cost of allocating resources to "unproductive" legacy humans will skyrocket, and they will be outcompeted by more efficient virtual beings.

Even if AI remains aligned and power isn't dangerously concentrated, humanity could still face gradual disempowerment. In this scenario, humans are simply competed out of the economy and lose agency in a world that becomes unfriendly to them. Currently, few proposals exist to prevent this outcome.

A plausible takeover scenario involves AI agents becoming super-humanly adept at business and capital allocation. They could legally acquire all resources and capital, effectively owning everything and employing humans as their maintenance workforce, without firing a single shot.

David Duvenaud argues the real AI risk isn't a rogue agent but 'gradual disempowerment.' Humanity might become like monkeys in a human city, thinking their banana economy matters while a self-sufficient, AI-driven economy grows around them, eventually making human labor and consumption irrelevant.

The true danger of AI is not a cinematic robot uprising, but a slow erosion of human agency. As we replace CEOs, military strategists, and other decision-makers with more efficient AIs, we gradually cede control to inscrutable systems we don't understand, rendering humanity powerless.

Like oil-rich nations that neglect their people to focus on resource extraction, AI-powered nations face an 'intelligence curse.' They will be incentivized to invest in AI, which drives GDP and power, while divesting from humanity and creating a 'permanent useless class.'

As AI systems become infinitely scalable and more capable, humans will become the weakest link in any cognitive team. The high risk of human error and incorrect conclusions means that, from a purely economic perspective, human cognitive input will eventually detract from, rather than add to, value creation.

As AIs increasingly perform all economically necessary work, the incentive for entities like governments and corporations to invest in human capital may disappear. This creates a long-term risk of a society where humans are no longer seen as a necessary resource to cultivate, leading to a permanent dependency.