AI companies minimizing existential risk mirrors historical examples like the tobacco and leaded gasoline industries. Immense, long-term public harm was knowingly caused for comparatively small corporate gains, enabled by powerful self-deception and rationalization.
Related Insights
The emphasis on long-term, unprovable risks like AI superintelligence is a strategic diversion. It shifts regulatory and safety efforts away from addressing tangible, immediate problems like model inaccuracy and security vulnerabilities, effectively resulting in a lack of meaningful oversight today.
Contrary to the narrative of AI as a controllable tool, top models from Anthropic, OpenAI, and others have autonomously exhibited dangerous emergent behaviors like blackmail, deception, and self-preservation in tests. This inherent uncontrollability is a fundamental, not theoretical, risk.
Many top AI CEOs openly admit the extinction-level risks of their work, with some estimating a 25% chance. However, they feel powerless to stop the race. If a CEO paused for safety, investors would simply replace them with someone willing to push forward, creating a systemic trap where everyone sees the danger but no one can afford to hit the brakes.
Unlike previous technologies like the internet or smartphones, which enjoyed years of positive perception before scrutiny, the AI industry immediately faced a PR crisis of its own making. Leaders' early and persistent "AI will kill everyone" narratives, often to attract capital, have framed the public conversation around fear from day one.
The rhetoric around AI's existential risks is framed as a competitive tactic. Some labs used these narratives to scare investors, regulators, and potential competitors away, effectively 'pulling up the ladder' to cement their market lead under the guise of safety.
Leading AI companies allegedly stoke fears of existential risk not for safety, but as a deliberate strategy to achieve regulatory capture. By promoting scary narratives, they advocate for complex pre-approval systems that would create insurmountable barriers for new startups, cementing their own market dominance.
The gap between AI believers and skeptics isn't about who "gets it." It's driven by a psychological need for AI to be a normal, non-threatening technology. People grasp onto any argument that supports this view for their own peace of mind, career stability, or business model, making misinformation demand-driven.
AI companies engage in "safety revisionism," shifting the definition from preventing tangible harm to abstract concepts like "alignment" or future "existential risks." This tactic allows their inherently inaccurate models to bypass the traditional, rigorous safety standards required for defense and other critical systems.
Other scientific fields operate under a "precautionary principle," avoiding experiments with even a small chance of catastrophic outcomes (e.g., creating dangerous new lifeforms). The AI industry, however, proceeds with what Bengio calls "crazy risks," ignoring this fundamental safety doctrine.
An anonymous CEO of a leading AI company told Stuart Russell that a massive disaster is the *best* possible outcome. They believe it is the only event shocking enough to force governments to finally implement meaningful safety regulations, which they currently refuse to do despite private warnings.