Significant opportunity exists in re-architecting how AI models work. Instead of building ever-larger single models, the focus is shifting to creating networks of smaller, specialized models that collaborate, which can drastically reduce the cost per token produced.

Liquid AI's early, highly effective non-linear models faced a major scaling bottleneck. Non-linear relationships are difficult to "tensorize"—convert from sequential to parallel computations—which is essential for GPU efficiency. This is why linear systems like state-space models scale more easily.

Overly structured, workflow-based systems that work with today's models will become bottlenecks tomorrow. Engineers must be prepared to shed abstractions and rebuild simpler, more general systems to capture the gains from exponentially improving models.

AI development history shows that complex, hard-coded approaches to intelligence are often superseded by more general, simpler methods that scale more effectively. This "bitter lesson" warns against building brittle solutions that will become obsolete as core models improve.

Resist building complex, multi-agent systems from day one. Instead, start with a single agent and build its skills based on actual workflows. Add sub-agents only when a clear productivity need arises. This approach is more effective than scaling for what looks impressive.

The path to robust AI applications isn't a single, all-powerful model. It's a system of specialized "sub-agents," each handling a narrow task like context retrieval or debugging. This architecture allows for using smaller, faster, fine-tuned models for each task, improving overall system performance and efficiency.

As AI models scale, their optimal architecture changes. Smaller models benefit from architectural "biases" like gating for efficiency. However, at massive scale (trillions of parameters), unstructured architectures like Transformers, which rely on simple matrix multiplication, become superior because they scale with fewer constraints.

Chai Discovery's core philosophy is a direct application of "The Bitter Lesson" to biotech. They prioritize scaling compute, data, and simple models over creating complex, bespoke biological modules, betting that general-purpose learning methods will outperform human-engineered ones at scale.

Early on, Google's Jules team built complex scaffolding with numerous sub-agents to compensate for model weaknesses. As models like Gemini improved, they found that simpler architectures performed better and were easier to maintain. The complex scaffolding was a temporary crutch, not a sustainable long-term solution.