Top AI researchers currently wield significant influence, able to force policy reversals at labs like Anthropic because their talent is indispensable. However, this power is temporary. Once recursive self-improvement (RSI) becomes effective, the models themselves will drive progress, concentrating power solely with leadership and diminishing researchers' leverage.

The tech industry now has two distinct classes of labor. In AI-native companies like Anthropic, elite researchers have immense power, dictating strategy and leaving eight-figure stock packages. In contrast, at traditional tech companies like Block, non-AI employees have become fungible, with management holding unprecedented leverage to enact deep cuts.

The belief in near-term recursive self-improving AI creates urgency for top researchers. They are leaving established companies to start new labs, aiming to secure massive capital and market position before the window of opportunity to build foundational models closes.

Over 1,300 researchers from OpenAI, Google, and Anthropic are urging government intervention because they believe AI systems are on the verge of automating their own R&D. This could lead to an uncontrollable acceleration in AI capabilities beyond human understanding.

CEOs claim they cannot slow down due to competitive pressure and antitrust laws. Garrison Lovely proposes that if researchers form unions focused on safety, these unions could legally coordinate a collective "pacing of the frontier" across different labs, a move protected under labor law.

The concept of Recursive Self-Improvement (RSI), where AI models help train the next generation, has created significant anxiety among AI researchers themselves. The conversation has evolved from AI automating software engineers to researchers questioning if their own roles will soon be obsolete.

The ultimate goal for leading labs isn't just creating AGI, but automating the process of AI research itself. By replacing human researchers with millions of "AI researchers," they aim to trigger a "fast takeoff" or recursive self-improvement. This makes automating high-level programming a key strategic milestone.

A key source of power for AI labs in government negotiations is the credible threat that their top researchers—a vital and mobile constituency—will revolt or quit if forced to comply with certain demands.

The key safety threshold for labs like Anthropic is the ability to fully automate the work of an entry-level AI researcher. Achieving this goal, which all major labs are pursuing, would represent a massive leap in autonomous capability and associated risks.