In developing the X-Cell model, Xaira found a clear hierarchy of impact. The quality, scale, and causal nature of the training data provided the most significant performance boost, followed by the choice of AI architecture (e.g., diffusion vs. autoregressive), and lastly, the integration of prior biological knowledge.

Diffusion models work on a continuous medium like an image by adding noise until it's unrecognizable, then training a model to reverse the process. This holistic, denoising method is fundamentally different from autoregressive models like large language models, which predict data one token at a time.

While GANs failed for protein systems, diffusion models became the key primitive. Now, the frontier of diffusion research is in specialized scientific areas like 3D structure prediction, surpassing the innovation seen in more mainstream AI applications like image generation.

Standard AI models trained on public, observational biological data excel at descriptive tasks but underperform even linear models on causal predictions. To predict cellular responses to drug-like perturbations, models must be trained specifically on causal data generated from targeted experiments.

Modern protein models use a generative approach (diffusion) instead of regression. Instead of predicting one "correct" structure, they model a distribution of possibilities. This better handles molecular dynamism and avoids averaging between multiple valid states, which is a flaw of regression models.

LLMs operate autoregressively, making one decision (token) at a time without seeing the full problem space. This can lead to hallucinations or dead ends. EBMs are non-autoregressive, allowing them to see all possible routes simultaneously and select an optimal path, much like having a bird's-eye view of a map to avoid a hole in the road.

A major misconception is that general-purpose Large Language Models (LLMs) can be readily applied to complex biological problems. Biological data, like RNA sequencing, constitutes a unique language that requires custom-built foundation models, not simply fine-tuning of existing LLMs.

Simple linear models fail to generalize to new cell types because gene functions are highly context-dependent. Some "housekeeping" genes have universal effects, but many others behave differently in various cellular environments. A sophisticated, nonlinear AI model is required to capture these context-specific interactions.

When developing their Tario transformer model, Noetik discovered a key scaling behavior: larger, autoregressive models only outperform smaller ones when given a longer context window (i.e., seeing more tissue at once). This suggests that capturing broader spatial relationships is critical for learning complex biological patterns.