When a company like prediction market Kalshi fines users for insider trading, it highlights a broader regulatory vacuum. Relying on companies to police themselves is an unsustainable model and an indictment of the lack of effective government oversight.

Proposed self-regulatory bodies for AI safety have a built-in flaw: they are incentivized to be overly restrictive. They face all the blame for safety failures but get no credit for economic gains from innovation, leading to a natural bias that stifles progress.

When companies like OpenAI and Anthropic pull products due to risk, it's a clear signal that they are unable to self-govern. This action is interpreted as a plea for government oversight, as relying on the social conscience of a few CEOs is an unsustainable model.

Leading AI labs OpenAI and Anthropic came close to a formal agreement to perform safety tests on each other's models but the deal was ultimately abandoned. This failure of industry self-regulation indicates that despite public calls for accountability, internal competition and complexity are preventing proactive measures, likely forcing government to step in.

Former OpenAI researcher Jerry Twerk argues against a central regulatory body for AI safety. He proposes that market forces are more effective, as competing labs have a strong financial incentive to audit each other's models, expose vulnerabilities, and publicize safety flaws, creating a self-policing ecosystem.

Major AI companies are not solely seeking to stifle competition with regulation. They are also signaling an inability to self-regulate amidst intense competitive pressure, effectively asking external bodies to impose a mandatory safety floor that applies to everyone.

The debate over AI regulation often gets bogged down in technical complexity. A simpler, powerful argument is that nearly every other impactful technology—from cars and planes to food and medicine—requires pre-market safety validation. AI, with its greater potential risks, should be no different.

Proposing a self-regulatory body modeled after FINRA for AI is seen as a deceptive tactic. Critics argue it's not truly "self-regulating" but a fig leaf for a new, slow-moving government agency that will implement pre-release testing and approvals, ultimately creating a "DMV for AI" that stifles innovation.

The creation of a self-regulatory body (SAFA) by the three most powerful AI labs raises significant concerns about regulatory capture. Critics worry the incumbents will establish stringent standards that are difficult for smaller players to meet, thereby cementing their market leadership.