Mark Zuckerberg argues that trying to stop the spread of dangerous information from AI is futile. A more practical and effective policy approach is to focus on controlling access to the physical materials (e.g., chemicals) needed to act on that information.

Simple refusal mechanisms in AI models are easily bypassed by motivated actors. Effective biosecurity requires deeper interventions, such as curating training data to exclude sensitive biological information or implementing strict access controls for the most powerful models, ensuring they aren't publicly available.

Top AI labs and biotech firms are urging the US government to mandate screening for nucleic acid synthesis orders. This pragmatic approach targets a concrete threat—AI-assisted bioweapon creation—rather than abstract superintelligence risks.

A global AI safety regime should learn from nuclear arms control by focusing on the physical infrastructure that enables strategic capabilities. Instead of just seeking promises, it should aim to control access to chokepoints like advanced chip manufacturing and the massive data centers required for frontier models.

Top AI labs are calling for mandatory government screening of nucleic acid synthesis, highlighting that the current voluntary system only covers a self-reported 80% of the industry. This proactive call for regulation signals that industry leaders see self-policing as insufficient to prevent AI-accelerated bioterrorism.

Vitalik Buterin suggests that slowing AI progress to buy time for safety is a valid goal. He argues the most feasible and least dystopian method is to limit hardware production. Since chip manufacturing is already highly centralized, it presents a control point that avoids more invasive, freedom-restricting measures.

The danger of AI creating harmful proteins is not in the digital design but in its physical creation. A protein sequence on a computer is harmless. The critical control point is the gene synthesis process. Therefore, biosecurity efforts should focus on providing advanced screening tools to synthesis providers.

Instead of trying to control open-source AI models, which is intractable, the proposed strategy is to control the small, expensive-to-produce functional datasets they train on. This preserves the beneficial open-source ecosystem while preventing the dissemination of dangerous capabilities like viral design.

The US nuclear weapons industry operates as a hybrid: the government owns the IP and facilities, but private contractors like Honeywell and Boeing operate them and build delivery systems. This established public-private partnership model could be applied to manage the risks of powerful, privately-developed AI.