We scan new podcasts and send you the top 5 insights daily.
Alex Rives notes that with infinite data, model architectures converge, but cellular biology is far from that limit. He says inductive bias therefore matters most when data is scarce, and today's transformer lacks the multi-scale structure needed to represent molecular and tissue levels in practice.
In developing the X-Cell model, Xaira found a clear hierarchy of impact. The quality, scale, and causal nature of the training data provided the most significant performance boost, followed by the choice of AI architecture (e.g., diffusion vs. autoregressive), and lastly, the integration of prior biological knowledge.
Alex Rives argues current virtual cell models fail because biology's measuring tools cannot observe the fine-grained causal structure models need. He says better architectures help, but Biohub is investing heavily in new measurement modalities and data generation because observation limits are the deeper bottleneck first.
Unlike text-based LLMs where simply increasing parameter count works, Verge Labs found the biggest AI performance gains in biology come from scaling data modalities—adding new types of data like proteomics and imaging. Fusing different data sources is more critical than just making the model bigger.
The primary obstacle to creating sophisticated AI models of cells isn't the AI itself, but the data. Existing datasets often perturb only one cellular variable at a time, failing to capture the complex interactions that arise from simultaneous changes. New platforms are needed to generate this multi-dimensional data.
Contrary to trends in other AI fields, structural biology problems are not yet dominated by simple, scaled-up transformers. Specialized architectures that bake in physical priors, like equivariance, still yield vastly superior performance, as the domain's complexity requires strong inductive biases.
A major misconception is that general-purpose Large Language Models (LLMs) can be readily applied to complex biological problems. Biological data, like RNA sequencing, constitutes a unique language that requires custom-built foundation models, not simply fine-tuning of existing LLMs.
Guardant's co-CEO is skeptical of many AI-in-biology efforts because the underlying public data is often 'very under sampled.' These low-resolution datasets miss the rare signals that differentiate cells, causing AI models to learn from noise rather than true biological drivers of disease.
Simple linear models fail to generalize to new cell types because gene functions are highly context-dependent. Some "housekeeping" genes have universal effects, but many others behave differently in various cellular environments. A sophisticated, nonlinear AI model is required to capture these context-specific interactions.
When developing their Tario transformer model, Noetik discovered a key scaling behavior: larger, autoregressive models only outperform smaller ones when given a longer context window (i.e., seeing more tissue at once). This suggests that capturing broader spatial relationships is critical for learning complex biological patterns.
Alex Rives says language captures centuries of biological knowledge but not empirical facts that only direct experiments reveal. He notes protein models learn features for unknown biology that lacks names, so biology may need its own separate latent space rather than mere arm's-length tool calls.