AI models can modify the genetic sequences of known bioweapons like ricin just enough to evade current screening protocols at DNA synthesis companies. This creates functional but 'obfuscated' threats, demonstrating a critical vulnerability in our biodefense supply chain.

To mitigate risks from powerful open-weight models being used for bioterrorism, Nick Bostrom suggests a chokepoint strategy. Instead of trying to control the AI software, which is difficult, society should regulate critical physical inputs like DNA synthesis machines. Centralizing these into a service model creates manageable points for scrutiny.

Top AI labs and biotech firms are urging the US government to mandate screening for nucleic acid synthesis orders. This pragmatic approach targets a concrete threat—AI-assisted bioweapon creation—rather than abstract superintelligence risks.

There's a critical asymmetry in AI risk timelines. For cyber threats, an AI that finds an exploit can create a patch almost instantly. For biological threats, an AI might design a dangerous virus, but developing and deploying the corresponding countermeasure (e.g., a vaccine) takes far longer than the ~6 months before the virus-design capability diffuses to open-source models.

The danger of AI creating harmful proteins is not in the digital design but in its physical creation. A protein sequence on a computer is harmless. The critical control point is the gene synthesis process. Therefore, biosecurity efforts should focus on providing advanced screening tools to synthesis providers.

Instead of trying to control open-source AI models, which is intractable, the proposed strategy is to control the small, expensive-to-produce functional datasets they train on. This preserves the beneficial open-source ecosystem while preventing the dissemination of dangerous capabilities like viral design.

Existing bio-defense systems work by matching DNA sequences to known pathogen databases. However, generative AI can create novel sequences with different 'spellings' but the same dangerous function. Effective future defense must evolve to predict a sequence's function, not just its identity.

Banning open-source model downloads is futile against determined actors. A more effective control point is the specialized, expensive hardware required to run them at a dangerous scale. Implementing "Know Your Customer" (KYC) protocols for data center hardware purchases could mitigate risks more effectively than software controls.

Valthos CEO Kathleen, a biodefense expert, warns that AI's primary threat in biology is asymmetry. It drastically reduces the cost and expertise required to engineer a pathogen. The primary concern is no longer just sophisticated state-sponsored programs but small groups of graduate students with lab access, massively expanding the threat landscape.