Proposed self-regulatory bodies for AI safety have a built-in flaw: they are incentivized to be overly restrictive. They face all the blame for safety failures but get no credit for economic gains from innovation, leading to a natural bias that stifles progress.

Leading AI labs OpenAI and Anthropic came close to a formal agreement to perform safety tests on each other's models but the deal was ultimately abandoned. This failure of industry self-regulation indicates that despite public calls for accountability, internal competition and complexity are preventing proactive measures, likely forcing government to step in.

The order's voluntary framework for pre-release model testing mirrors agreements major AI labs already have with the Department of Commerce. It reflects current industry behavior rather than imposing new, substantive regulations, failing to meet public calls for stronger government oversight.

The idea of nations collectively creating policies to slow AI development for safety is naive. Game theory dictates that the immense competitive advantage of achieving AGI first will drive nations and companies to race ahead, making any global regulatory agreement effectively unenforceable.

Major AI companies are not solely seeking to stifle competition with regulation. They are also signaling an inability to self-regulate amidst intense competitive pressure, effectively asking external bodies to impose a mandatory safety floor that applies to everyone.

The debate over AI regulation often gets bogged down in technical complexity. A simpler, powerful argument is that nearly every other impactful technology—from cars and planes to food and medicine—requires pre-market safety validation. AI, with its greater potential risks, should be no different.

Direct, detailed government regulation of AI in the U.S. is unlikely to be effective. A better model is a self-regulatory organization like FINRA, where the government sets broad risk tolerance levels, and an industry body creates and enforces specific technical rules.

Proposing a self-regulatory body modeled after FINRA for AI is seen as a deceptive tactic. Critics argue it's not truly "self-regulating" but a fig leaf for a new, slow-moving government agency that will implement pre-release testing and approvals, ultimately creating a "DMV for AI" that stifles innovation.

Instead of direct regulation, the government could act as a reluctant mediator for AI safety. By setting a deadline for labs to form their own collaborative safety pact, it creates a powerful incentive: if they fail, the government will impose 'heavy-handed' and likely suboptimal regulations, an outcome all parties want to avoid.