Mark Zuckerberg argues that trying to stop the spread of dangerous information from AI is futile. A more practical and effective policy approach is to focus on controlling access to the physical materials (e.g., chemicals) needed to act on that information.

Top AI labs and biotech firms are urging the US government to mandate screening for nucleic acid synthesis orders. This pragmatic approach targets a concrete threat—AI-assisted bioweapon creation—rather than abstract superintelligence risks.

There's a critical asymmetry in AI risk timelines. For cyber threats, an AI that finds an exploit can create a patch almost instantly. For biological threats, an AI might design a dangerous virus, but developing and deploying the corresponding countermeasure (e.g., a vaccine) takes far longer than the ~6 months before the virus-design capability diffuses to open-source models.

While creating a bioweapon may be cheaper than defending against it, biology is inherently defense-dominant. Pathogens are vulnerable to physical barriers, filtration, heat, and UV light. Their small size is a weakness, and unlike intelligent adversaries, they cannot strategically penetrate defenses, giving defenders a fundamental advantage.

Instead of releasing new AI models to everyone simultaneously, a better strategy is providing early, privileged access to trusted defenders like vaccine developers. This allows them to build countermeasures and create a 'defensive uplift' advantage before malicious actors can exploit new capabilities.

Open-source frontier models will likely democratize biological design, escalating the potential distribution of dangerous pathogens. Rather than relying on elusive software restrictions, defense must focus on physical bottlenecks by converting localized DNA synthesis machines into centralized, strictly monitored commercial services.

Zvi Moshwitz argues against the "X is not the bottleneck" framing for bioweapon risk. In a multi-step process, solving any single step with AI makes the entire chain easier to complete. Removing the AI knowledge barrier is like giving away a key part of the blueprint to bad actors, dramatically increasing the risk.

The underlying AI architecture for creating novel biological sequences is also highly effective at identifying dangerous ones. This dual-use nature means the team building design capabilities is best suited to build defense tools, as the models are fundamentally the same.

"Gain-of-function" research, intended to develop countermeasures for dangerous pathogens, uses the same methods as offensive bioweapon creation. This paradox means that efforts to defend against biological threats inherently increase the risk of creating them, whether deliberately or by accident.