The future of enterprise AI isn't choosing one provider. Instead, companies will use a "composable model" approach, routing queries to a combination of powerful frontier models and their own fine-tuned open-source models. This strategy, dubbed the "council of LLMs," optimizes for cost, performance, and specialization on proprietary data.

Glean's co-founder argues that most enterprise tasks don't require expensive frontier models. Open-source alternatives are now capable enough for the vast majority of use cases. The primary adoption driver has shifted from data privacy to pure cost savings, as enterprises seek to control skyrocketing AI bills.

Early enterprise AI adoption mirrored the initial, inefficient use of AWS, with rampant experimentation. Now, companies are maturing, learning to apply AI strategically, much like a savvy Costco shopper who targets specific items instead of wandering every aisle. This shift involves using cheaper or open-source models for simpler tasks and reserving frontier models for high-value problems.

For most enterprise tasks, massive frontier models are overkill—a "bazooka to kill a fly." Smaller, domain-specific models are often more accurate for targeted use cases, significantly cheaper to run, and more secure. They focus on being the "best-in-class employee" for a specific task, not a generalist.

For typical enterprise tasks like code migration, using an optimized control plane with an open-source model can be over 16 times cheaper than using a frontier model like Claude Opus. While it may be slower, the massive cost savings make it a compelling business alternative.

As enterprises scale AI, the high inference costs of frontier models become prohibitive. The strategic trend is to use large models for novel tasks, then shift 90% of recurring, common workloads to specialized, cost-effective Small Language Models (SLMs). This architectural shift dramatically improves both speed and cost.

The greatest value in AI won't be captured by frontier labs alone. Instead, companies in the "applied layer" are incentivized to build routing systems that use expensive frontier models for high-level orchestration while deploying cheaper open-source models for bulk tasks, creating a more efficient, barbell-shaped cost structure.

The smartest 'AI-pilled' companies adopt a two-tiered model strategy. They use expensive, frontier models for internal, high-leverage tasks like creating new knowledge and optimizing processes. However, they use cheaper, open-weight models in the 'bill of materials' for the customer-facing product to manage costs effectively.

An emerging rule from enterprise deployments is to use small, fine-tuned models for well-defined, domain-specific tasks where they excel. Large models should be reserved for generic, open-ended applications with unknown query types where their broad knowledge base is necessary. This hybrid approach optimizes performance and cost.