Leading AI labs OpenAI and Anthropic came close to a formal agreement to perform safety tests on each other's models but the deal was ultimately abandoned. This failure of industry self-regulation indicates that despite public calls for accountability, internal competition and complexity are preventing proactive measures, likely forcing government to step in.

The US government's new AI safety testing framework is secret, with details withheld even from uninvited AI companies. This approach prioritizes maximum flexibility for the government but creates 'minimum knowability' for the industry, hindering planning and fostering distrust.

The AI industry has no third-party verification; labs self-report performance on bias and accuracy via blog posts. Campbell Brown likens this to banks auditing themselves, arguing it creates an accountability vacuum and undermines public trust in a foundational technology.

Former OpenAI researcher Jerry Twerk argues against a central regulatory body for AI safety. He proposes that market forces are more effective, as competing labs have a strong financial incentive to audit each other's models, expose vulnerabilities, and publicize safety flaws, creating a self-policing ecosystem.

The AI auditing field risks a race to the bottom, where firms offer cheap, superficial audits. To ensure accountability, legislation must require auditors to publicly post their methodologies and code, allowing the community to scrutinize their work and establish robust standards.

By developing its AI safety framework in closed-door meetings and restricting access to written details, the White House is creating a 'black box' system. Critics argue this lack of transparency actively damages public trust—the very thing the framework is supposed to build—and creates uncertainty even for participating labs.

An open-source AI ban won't be explicit. Instead, a regulatory body influenced by incumbent closed-model companies will set "fair" safety standards. These standards will require monitoring mechanisms technologically inherent to closed models but impossible for decentralized open-source models to implement, regulating them out of existence.

The creation of a self-regulatory body (SAFA) by the three most powerful AI labs raises significant concerns about regulatory capture. Critics worry the incumbents will establish stringent standards that are difficult for smaller players to meet, thereby cementing their market leadership.

Unlike past tech waves where companies resisted government oversight, today's AI leaders are actively inviting it. This is a strategic move to shape regulations in their favor, creating barriers to entry for smaller players and open-source competitors under the guise of safety and responsibility.