Daniel Ek suggests that instead of focusing on flawed metrics like training flops, AI regulation should consider the amount of compute power being used. Access to massive GPU clusters is a more durable chokepoint and a better indicator of potentially powerful, large-scale AI operations.

Unlike closed models accessed via API, open source models can be downloaded and fine-tuned locally for malicious purposes, such as hacking or bioweapons research. This offline training capability makes them fundamentally harder to regulate and monitor.

If Anthropic genuinely feared misuse, it would implement Know-Your-Customer (KYC) protocols to vet users. Its failure to do so, while simultaneously calling for broad industry regulation, suggests its real agenda is to hobble open-source rivals, not ensure safety.

Instead of trying to control open-source AI models, which is intractable, the proposed strategy is to control the small, expensive-to-produce functional datasets they train on. This preserves the beneficial open-source ecosystem while preventing the dissemination of dangerous capabilities like viral design.

Attempts to ban or sanction open-source AI will inevitably fail because software is easily distributed. As models become capable of running on-device, hardware-based leverage points like data centers will become irrelevant, making any ban unenforceable.

An open-source AI ban won't be explicit. Instead, a regulatory body influenced by incumbent closed-model companies will set "fair" safety standards. These standards will require monitoring mechanisms technologically inherent to closed models but impossible for decentralized open-source models to implement, regulating them out of existence.

Anthropic's CEO clarified his stance is not for banning open-weight AI models. Instead, he advocates for specific policies like enforcing chip export controls, stopping large-scale model distillation, and requiring safety testing for all powerful models, both open and closed. This is a more nuanced position than a simple pro-regulation stance.

Unlike physical goods or closed software, China's open-weight AI models can be downloaded and distributed freely by anyone. Once the model is released, governments cannot easily enforce bans or sanctions, as the "genie is out of the bottle," posing a significant new challenge to digital trade regulation.

Western attempts to regulate AI are largely performative because powerful, open-source models already exist, particularly from China. Imposing draconian restrictions will only disarm compliant actors in the West, while malicious actors worldwide will continue to leverage the unrestricted models that are already publicly available.