David Sacks argues that for a self-regulatory organization (SRO) to be effective and avoid capture by incumbents, it must have broad representation. This means including voices from startups and the open-source community, not just the three largest and most powerful AI labs.

Large AI firms like Anthropic are advocating for stringent government regulation under the guise of safety. However, these proposed rules also serve to raise the barrier to entry, making it more difficult for cheaper, open-source models and startups to compete, thus protecting the incumbents' market share.

The push for a self-regulatory body for AI, modeled on the financial industry's FINRA, has stalled amid fears it could stifle competition. Critics argue it's a "Trojan horse" that would allow frontier labs to impose compliance standards that are impossible for smaller, open-source developers to meet, effectively protecting incumbents.

Leading AI companies, like Anthropic, are accused of manufacturing fear about AI's dangers to push for a pre-approval system for new models. This creates a regulatory moat that protects their market lead by boxing out smaller startups that can't navigate the bureaucracy.

Proposed self-regulatory bodies for AI safety have a built-in flaw: they are incentivized to be overly restrictive. They face all the blame for safety failures but get no credit for economic gains from innovation, leading to a natural bias that stifles progress.

Attempts by professional guilds, like lawyers or accountants, to mandate human involvement through legislation are doomed to fail. Vertically-integrated AI competitors will offer cheaper, alternative services, and the widespread availability of open-source models will make such protectionist regulations unenforceable.

The narrative of AI doom isn't just organic panic. It's being leveraged by established players who are actively seeking "regulatory capture." They aim to create a cartel that chokes off innovation from startups right from the start.

Leading AI companies allegedly stoke fears of existential risk not for safety, but as a deliberate strategy to achieve regulatory capture. By promoting scary narratives, they advocate for complex pre-approval systems that would create insurmountable barriers for new startups, cementing their own market dominance.

Leaders from Anthropic and DeepMind have voiced support for creating a self-regulatory organization (SRO) for AI, modeled after the financial industry's FINRA. Such a body could establish standards and enforce rules more quickly than government, with real power to decertify non-compliant companies.