Enterprises are currently overspending on tokens by sending all queries to the most powerful LLMs. A new software category will emerge to intelligently route requests to smaller, cheaper models when possible, creating a critical efficiency and cost-saving layer between companies and foundational model providers.

Prompted by the risk of government shutdowns, architectural approaches like OpenRouter's Fusion API are shifting from being cost-optimization tools to essential infrastructure for resilience. This approach ensures continuity by fanning out prompts to multiple models, mitigating the risk of a single point of failure.

Sophisticated model routers do more than route queries to the cheapest AI model. Palantir's Evolve tool also automatically optimizes prompts for the target model, a dual approach that can reduce token consumption by 60% and overall compute costs by up to 97% for specific tasks.

Advanced AI architectures will use small, fast, and cheap local models to act as intelligent routers. These models will first analyze a complex request, formulate a plan, and then delegate different sub-tasks to a fleet of more powerful or specialized models, optimizing for cost and performance.

Instead of relying on one powerful model for all tasks, the leading strategy is 'smart routing'—using a panel of models and directing each task to the most appropriate one. This compound architecture demonstrably beats single frontier models on both cost and performance.

Instead of relying on a single large AI model, companies are adopting "model orchestration" to control costs. This involves using a router to send prompts to the most appropriate model based on the task, often cascading from cheap, small models to more expensive ones only when necessary.

Companies are building intelligent systems that analyze a user's prompt and automatically route it to the most cost-effective model that can handle the task. This avoids using expensive frontier models for simple requests, with some companies like Coinbase successfully keeping costs flat despite exponential usage growth.

The recent focus on model routers signals a maturation of enterprise AI strategy. The initial "growth at all costs" phase, which encouraged rampant employee use ("token maxing"), is giving way to a new era of cost optimization and demonstrating clear ROI on AI investments.

To prevent AI agent usage costs from spiraling, GitHub expects the solution will be intelligent model routing. These systems will automatically select the most efficient and cost-effective AI model for a given task, such as using a cheap model for simple refactoring instead of a powerful, expensive one.