Enterprises are currently overspending on tokens by sending all queries to the most powerful LLMs. A new software category will emerge to intelligently route requests to smaller, cheaper models when possible, creating a critical efficiency and cost-saving layer between companies and foundational model providers.

To optimize AI costs and sustainability, UBS employs a "model garden" with various frontier and smaller models. An internal AI system then routes employee questions to the most appropriate, cost-effective model, preventing the wasteful use of powerful, expensive LLMs for simple, non-frontier problems.

Companies are discovering they're overpaying for AI by using powerful models for mundane tasks. They will increasingly adopt routers that intelligently direct queries to the most cost-effective model. This move will drive down costs and commoditize the AI model layer.

Companies are building intelligent systems that analyze a user's prompt and automatically route it to the most cost-effective model that can handle the task. This avoids using expensive frontier models for simple requests, with some companies like Coinbase successfully keeping costs flat despite exponential usage growth.

A sophisticated gateway that routes queries to different models based on complexity is key to managing AI costs. Simple tasks go to cheap, open-source models, while difficult ones use the frontier. This "expert pattern" allows token usage to rise while keeping costs flat.

Companies like Meta and Ramp are building AI routers to automatically send simple tasks to cheaper models. This trend shows the enterprise AI market is maturing past a 'one-model-fits-all' approach, focusing instead on cost management and operational efficiency by treating models as a commodity portfolio.

To prevent AI agent usage costs from spiraling, GitHub expects the solution will be intelligent model routing. These systems will automatically select the most efficient and cost-effective AI model for a given task, such as using a cheap model for simple refactoring instead of a powerful, expensive one.

A production AI agent performs tasks of varying difficulty. Forcing all requests through a single, expensive frontier model is inefficient. A better architecture routes tasks to the most appropriate model: small, cheap open models for high-volume, low-difficulty work like retrieval, reserving the costly frontier API only for high-stakes reasoning where it matters.

To control inference costs, companies are implementing model routing systems. They differentiate between expensive tokens from frontier models for complex reasoning and cheaper tokens from fine-tuned open-source models for simpler workflow tasks. This tiered approach optimizes both performance and budget, avoiding "token maxing."