Anthropic's call for third-party evaluators is undermined by its choice of Meter, an organization critics say is deeply intertwined with both Anthropic and OpenAI. With a shared history, personnel, and ecosystem, Meter's ability to act as a truly independent referee is being heavily questioned.

Frontier AI labs now actively call for third-party verification. This is a strategic response to a significant public "trust deficit" and the realization they cannot self-certify their way to broad adoption and social license.

Abstract theory from outside an AI lab is unlikely to be adopted due to immense internal implementation constraints. To be useful, external research must provide a concrete solution, a new evaluation, or a clear metric that can be easily integrated into a complex, fragile development pipeline.

The AI safety auditing ecosystem is fundamentally flawed. Auditors get minimal access (e.g., three days with Astra), are underfunded, constantly lose talent to the very labs they audit, and face legal and operational hurdles, making effective, independent oversight nearly impossible.

The AI industry has no third-party verification; labs self-report performance on bias and accuracy via blog posts. Campbell Brown likens this to banks auditing themselves, arguing it creates an accountability vacuum and undermines public trust in a foundational technology.

The proper division of labor in AI safety is for the government to define what it's afraid of—the "rules" against biohacking or cyber hacking—and enforce them. The government is not equipped to perform the complex, fast-moving technical work of evaluating if models can break those rules, which should be handled by specialized third-party evaluators.

External investigators into AI incidents, like at OpenAI, face a power imbalance. Their access is limited, and they must stay on good terms with labs to be invited back, compromising the candor of their reports and hindering true oversight.

Auditing frontier AI models cannot follow a traditional, once-a-year checklist model. Due to rapid development, verifiers must be deeply embedded with labs, working "hip-to-hip" to continuously assess systems from pre-deployment through their entire lifecycle.

Critics argue that proposed third-party evaluators, such as Meter, lack true independence. Their staff often includes former employees from the very AI labs they would audit (OpenAI, Anthropic), creating a "revolving door" that raises questions about conflicts of interest.