Bill Gates’ warning about the turbulent societal impact of AI is especially potent because it comes from an industry pioneer. His assertion that tech companies cannot be trusted to self-regulate directly contradicts the narrative pushed by current tech leaders, adding significant weight to calls for external government oversight.

Mark Zuckerberg frames teen safety as a prisoner's dilemma requiring government intervention to force competitors like TikTok to adopt similar standards. Conversely, he argues the market will self-correct for AI safety, revealing a strategic, rather than principled, approach to regulation.

A significant barrier to voluntary safety pacts among AI companies is antitrust law. An agreement to slow development could be prosecuted as illegal anti-competitive collusion, as it would limit the technology available to consumers. This makes government-led frameworks essential for any coordinated industry action.

The proper division of labor in AI safety is for the government to define what it's afraid of—the "rules" against biohacking or cyber hacking—and enforce them. The government is not equipped to perform the complex, fast-moving technical work of evaluating if models can break those rules, which should be handled by specialized third-party evaluators.

Pritzker says his AI cabinet pairs technical experts with agency leaders and includes emergency management and state police. The goal is to discuss privacy, safety and efficiency together, rather than chasing efficiency alone before the state understands AI's dangers. He deliberately avoids plunging headlong and says experts come from universities.

Federal and state governments are massive customers of technology. Instead of relying solely on legislation, they can use their procurement power to enforce AI safety and ethical standards. By setting strict purchasing requirements, they can compel companies to build more responsible products.

Cohere's leadership warns that setting AI regulations and safety testing procedures behind closed doors with a small cabal of industry incumbents undermines public trust. Effective and trusted governance requires transparent public benchmarks, standard testing methodologies, and oversight conducted by independent third-party auditors rather than proprietary self-regulation.

Major AI companies are not solely seeking to stifle competition with regulation. They are also signaling an inability to self-regulate amidst intense competitive pressure, effectively asking external bodies to impose a mandatory safety floor that applies to everyone.

Voluntary, self-policing agreements for the AI industry are ineffective political gestures. Critical sectors like aviation and finance require external regulation to ensure safety, a model that should apply to the rapidly growing AI industry.