Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Alex Rives says language captures centuries of biological knowledge but not empirical facts that only direct experiments reveal. He notes protein models learn features for unknown biology that lacks names, so biology may need its own separate latent space rather than mere arm's-length tool calls.

Related Insights

The next major AI breakthrough will come from applying generative models to complex systems beyond human language, such as biology. By treating biological processes as a unique "language," AI could discover novel therapeutics or research paths, leading to a "Move 37" moment in science.

Historically, biological discovery relied on harvesting naturally occurring proteins from bacteria or organisms like CRISPR and PCR. By training large language models on sequential protein data instead of natural text, Retro Biosciences and OpenAI demonstrated that bio-native models can generate completely new, functional human proteins at will. This transition shifts protein engineering from scrounging natural biology to an exponential, de novo generative paradigm.

The core philosophy behind ESMFold is that massive datasets and large transformer models can learn fundamental biological principles without needing built-in domain knowledge, applying Rich Sutton's "The Bitter Lesson" directly to bioinformatics.

Human minds struggle to grasp the vast complexity of biological systems. The guest argues that AI is the natural language for biology, just as mathematics is for physics, because AI models can capture the intricate, interconnected dynamics that are beyond human intuition.

The success of protein language models can be explained by Zellig Harris's 1954 linguistic theory. Just as a word's meaning is defined by its contexts, an amino acid's biological role is determined by the sequences it can appear in. The model learns this deep statistical structure, effectively learning biology.

A major misconception is that general-purpose Large Language Models (LLMs) can be readily applied to complex biological problems. Biological data, like RNA sequencing, constitutes a unique language that requires custom-built foundation models, not simply fine-tuning of existing LLMs.

Trained only on sequence prediction, ESM-C independently developed a hierarchical feature space mirroring decades of human scientific discovery. Its learned representations range from basic biochemical properties to complex, abstract functional concepts, all without prior biological knowledge.

Alex Rives expects biology to follow a pretraining-plus-elicitation recipe, but says reinforcement learning must be verified in laboratories. Because biology is empirical, he envisions AI systems proposing real experiments, running them, learning from actual results, and recursively improving their biological models over repeated improvement cycles.

Biohub applies mechanistic interpretability to its protein language models. By analyzing the model's internal representations—learned from both known and unknown biology—researchers can uncover emergent biological principles. This turns the model from a black box predictor into an engine for scientific discovery itself.

Traditional science failed to create equations for complex biological systems because biology is too "bespoke." AI succeeds by discerning patterns from vast datasets, effectively serving as the "language" for modeling biology, much like mathematics is the language of physics.