Contrary to the belief that enterprises have unlimited budgets, they are focused on the ROI of their AI spend. As agentic workflows cause token bills to skyrocket, orchestration tools that intelligently route queries to the most cost-effective model for a given task are becoming essential infrastructure.

The most heated topic among Fortune 500 CIOs is no longer which AI model is most powerful, but how to manage unpredictable and soaring token costs. Companies are struggling to find the right strategies—from workload prioritization to user-based access tiers—to create a predictable cost model in a rapidly evolving tech landscape.

The AI industry has shifted from a subsidized model to a "token shortage" era. This forces all companies, from AI providers to enterprise users like Uber, to prioritize cost-effective usage. Business models are now usage-based, making architectural and financial efficiency paramount.

While early generative AI costs were negligible, the shift to complex, multi-step agentic workflows is causing a massive spike in token usage. This has elevated cost optimization and ROI from a minor concern to a C-suite priority for the first time.

Instead of relying on a single large AI model, companies are adopting "model orchestration" to control costs. This involves using a router to send prompts to the most appropriate model based on the task, often cascading from cheap, small models to more expensive ones only when necessary.

In response to budget blowouts from agentic AI, enterprises are moving beyond simple adoption to active cost management. A new "token efficiency" stack is emerging, featuring tactics like model routing to cheaper alternatives (e.g., DeepSeek) and custom post-trained models to reduce reliance on expensive foundation models.

Drawing a parallel to the microservices boom, enterprises will soon deploy thousands of AI agents, creating immense operational complexity. The most valuable future products will be those that, like Datadog for microservices, provide governance, monitoring, and orchestration for this sprawling agentic workforce.

As AI token consumption becomes a major budget item, companies are moving beyond using a single frontier model. Every organization will need a portfolio of models, including cheaper options for less complex tasks, to manage the "madness" of runaway costs.

Paralleling the cloud adoption curve, the current surge in AI spending will inevitably be followed by an 'optimization point.' Enterprises will shift from experimentation to efficiency, scrutinizing token usage and seeking to reduce costs, forcing AI providers to help them optimize.