We scan new podcasts and send you the top 5 insights daily.
Alex Rives points out ESM and AlphaFold trained across all evolution, not just human proteins. He says core biological machinery is conserved while programs are rewired, so predictive biological models can transfer from many organisms instead of relying on a single imperfect laboratory mouse proxy.
While de-duplicating protein databases helps learn diverse structures, subtle variations in similar sequences are essential for learning function, as a single mutation can be catastrophic. This justifies training models on massive, unclustered datasets to capture fine-grained functional determinants.
DE Shaw Research (DESRES) invested heavily in custom silicon for molecular dynamics (MD) to solve protein folding. In contrast, DeepMind's AlphaFold, using ML on experimental data, solved it on commodity hardware. This demonstrates data-driven approaches can be vastly more effective than brute-force simulation for complex scientific problems.
The core philosophy behind ESMFold is that massive datasets and large transformer models can learn fundamental biological principles without needing built-in domain knowledge, applying Rich Sutton's "The Bitter Lesson" directly to bioinformatics.
The ESM-C model's performance leap came from adding billions of "noisy" protein sequences from environmental samples. This vast, diverse dataset overcame the limitations of curated databases like Uniref, removing the data bottleneck and revealing clear scaling laws.
Models like AlphaFold don't solve protein folding from physics alone. They heavily rely on co-evolutionary data, where correlated mutations across species provide strong hints about which amino acids are physically close. This dramatically constrains the search space for the final structure.
Traditional protein engineering relies on slow, serial single-residue substitutions in wet labs to preserve stability. Machine learning models trained on evolutionary datasets can predict multiple coordinated amino acid changes simultaneously, cutting optimization rounds down from dozens to a handful.
Since DNA is the source code for RNA and proteins, a foundation model pre-trained on DNA can learn underlying biological principles that transfer across modalities. This allows a single model to tackle tasks that previously required specialized protein or RNA models.
Trained only on sequence prediction, ESM-C independently developed a hierarchical feature space mirroring decades of human scientific discovery. Its learned representations range from basic biochemical properties to complex, abstract functional concepts, all without prior biological knowledge.
Resvita Bio's approach isn't about creating proteins from scratch. Instead, they use machine learning to 'read the book of life comprehensively,' analyzing how different organisms have evolved to solve the same biological problem. This allows them to synthesize nature's best solutions into an ideal therapeutic protein.
Alex Rives says language captures centuries of biological knowledge but not empirical facts that only direct experiments reveal. He notes protein models learn features for unknown biology that lacks names, so biology may need its own separate latent space rather than mere arm's-length tool calls.