While de-duplicating protein databases helps learn diverse structures, subtle variations in similar sequences are essential for learning function, as a single mutation can be catastrophic. This justifies training models on massive, unclustered datasets to capture fine-grained functional determinants.

Training a language model to predict the next amino acid in a sequence forces it to learn the protein's 3D structure. To make accurate predictions, the model must understand an amino acid's physical microenvironment, effectively deriving 3D spatial relationships from 1D sequence data alone. This demonstrates emergent capabilities of LLMs in biology.

The primary bottleneck for creating powerful foundation models in biology is the lack of clean, large-scale experimental data—orders of magnitude less than what's available for LLMs. This creates a major opportunity for "data foundries" that use robotic labs to generate high-quality biological data at scale.

DE Shaw Research (DESRES) invested heavily in custom silicon for molecular dynamics (MD) to solve protein folding. In contrast, DeepMind's AlphaFold, using ML on experimental data, solved it on commodity hardware. This demonstrates data-driven approaches can be vastly more effective than brute-force simulation for complex scientific problems.

Modern protein models use a generative approach (diffusion) instead of regression. Instead of predicting one "correct" structure, they model a distribution of possibilities. This better handles molecular dynamism and avoids averaging between multiple valid states, which is a flaw of regression models.

The low-hanging fruit of applying AI to existing datasets is being picked. The next major leap forward will come not from slightly better models, but from creative strategies to generate entirely new datasets for unsolved problems like protein stability or in vivo effects.

Contrary to trends in other AI fields, structural biology problems are not yet dominated by simple, scaled-up transformers. Specialized architectures that bake in physical priors, like equivariance, still yield vastly superior performance, as the domain's complexity requires strong inductive biases.

Traditional methods like crystallography are slow and analyze purified proteins outside their native environment. A-muto's platform uses proteomics and AI to analyze thousands of protein conformations in living disease models, capturing a more accurate picture of disease biology and identifying novel targets.

AlphaFold reproduced static structures deposited in the Protein Data Bank rather than modeling complete protein dynamics or actual conformational ground states. Proteins are complex, disordered, and context-dependent, meaning biological understanding of dynamics remains an open challenge.