Author Garrison Lovely's thesis posits that AGI is better understood as a machine that manufactures labor itself, rather than a thinking entity. This framing shifts the core concern from philosophical consciousness to economic upheaval, where capital can directly substitute labor, cratering wages and upending economic models.

The narrative that AI could be catastrophic ('summoning the demon') is used strategically. It creates a sense of danger that justifies why a small, elite group must maintain tight control over the technology, thereby warding off both regulation and competition.

When AI leaders acknowledge even a small chance of human extinction, it forces a governmental response. The only historical precedent for managing private-sector tech with true existential risk (e.g., nuclear weapons) is nationalization, eliminating the middle ground for responsible private innovation.

AI leaders often message their technology with a dual warning: it will automate jobs and poses existential risks. This 'cursed microwave' pitch, as Noah Smith describes it, is a terrible value proposition that alienates the public and provides ammunition for regulators pushing to halt AI development.

The classic argument that technology always creates new jobs is flawed when applied to AGI. Previous inventions like the tractor automated a single sector. AGI, by its nature, automates all forms of human cognitive labor—from finance to programming—simultaneously, overwhelming society's capacity to retrain and adapt.

The most significant risk to AI development is not a technical challenge but a widespread public outcry from those whose jobs are displaced. This could lead to a "burn down OpenAI" mentality, resulting in crippling regulations that halt progress out of fear and sympathy for the displaced.

By constantly comparing AI's power to nuclear weapons, tech leaders are making a powerful argument against their own independence. If the technology is truly an existential threat, it logically follows that it should be government-controlled for national security, not managed by venture-backed startups.

The fundamental challenge of creating safe AGI is not about specific failure modes but about grappling with the immense power such a system will wield. The difficulty in truly imagining and 'feeling' this future power is a major obstacle for researchers and the public, hindering proactive safety measures. The core problem is simply 'the power.'

In a notable rhetorical shift, OpenAI now argues that as AI capability grows, the human role in setting direction, making trade-offs, and applying values becomes *more* critical, not less. This positions AI as a tool for augmentation rather than a vehicle for full automation.