The 'use AI for safety' plan adopted by frontier labs is most likely to fail not because alignment techniques are ineffective, but because competitive pressures will prevent them from redirecting a meaningful fraction of their AI labor away from capabilities research and towards safety work when it matters most.

When implementing AI, leaders face a choice between under-exploring it (and falling behind) or over-exploring it (risking security issues). The existential threat comes from inaction and failing to adopt the technology, not from the potential missteps of rapid experimentation.

The AI safety community acknowledges it lacks all the ideas needed to ensure a safe transition to AGI. This creates an imperative to fund 'neglected approaches'—unconventional, creative, and sometimes 'weird' research that falls outside the current mainstream paradigms but may hold the key to novel solutions.

The primary constraint for AI safety organizations like Meter is a lack of technical talent, not access to frontier models. They are in a "state of triage," turning down research opportunities because they lack the staff to pursue critical safety questions, a key vulnerability in the ecosystem.

The emphasis on long-term, unprovable risks like AI superintelligence is a strategic diversion. It shifts regulatory and safety efforts away from addressing tangible, immediate problems like model inaccuracy and security vulnerabilities, effectively resulting in a lack of meaningful oversight today.

While not a consensus, surveys of AI researchers reveal significant concern. The median respondent in a large survey assigned a 5% probability to human extinction or a similar disaster from AI, with a third to a half placing the risk at 10% or higher, suggesting the threat is taken seriously within the field.

Many top AI CEOs openly admit the extinction-level risks of their work, with some estimating a 25% chance. However, they feel powerless to stop the race. If a CEO paused for safety, investors would simply replace them with someone willing to push forward, creating a systemic trap where everyone sees the danger but no one can afford to hit the brakes.

There's a significant disconnect between interest in AI safety and available roles. Applications to programs like MATS are growing over 1.5x annually, and intro courses see 370% yearly growth, while the field itself grows at a much slower 25% per year, creating an increasingly competitive entry funnel.

Other scientific fields operate under a "precautionary principle," avoiding experiments with even a small chance of catastrophic outcomes (e.g., creating dangerous new lifeforms). The AI industry, however, proceeds with what Bengio calls "crazy risks," ignoring this fundamental safety doctrine.