AI labs profit from token generation, creating a "big token" economy that conflicts with enterprise budgets. The solution is to use a portfolio of models—large ones for complex tasks and smaller, cheaper ones for simple edits—to optimize the cost-performance ratio.

Enterprises are currently overspending on tokens by sending all queries to the most powerful LLMs. A new software category will emerge to intelligently route requests to smaller, cheaper models when possible, creating a critical efficiency and cost-saving layer between companies and foundational model providers.

As enterprise usage ramps up, LLM token bills threaten corporate budgets, creating intense pressure on CIOs and CFOs to reduce costs. Enterprises do not need frontier-grade intelligence for every workflow; near-frontier open-weight models that trail frontier models by 6 to 12 months can handle 90% of reasoning, coding, and agentic tasks at a third of the cost. This dynamic will force enterprises to downgrade non-critical workloads away from expensive closed models.

When building a new agent feature, the best practice is to use the most powerful model available to explore the full capability space. Only after establishing clear success criteria and evaluation frameworks should you optimize by using smaller, more cost-effective models.

To manage AI costs effectively, companies should avoid simply capping token usage, as this kills innovation. A better strategy is to build intelligent routers that assess a task's complexity and dynamically route it to the most appropriate model—powerful models for hard tasks, cheaper ones for simple tasks.

To foster breakthrough ideas, companies should initially provide engineers with unrestricted access to the most powerful AI models, ignoring costs. Optimization should only happen after an idea proves its value at scale, as early cost-cutting stifles creativity.

A sophisticated gateway that routes queries to different models based on complexity is key to managing AI costs. Simple tasks go to cheap, open-source models, while difficult ones use the frontier. This "expert pattern" allows token usage to rise while keeping costs flat.

For difficult, multi-step tasks, a more capable LLM can reach a solution with fewer iterations than a smaller model. Despite a higher per-token price, this efficiency can lead to a lower total token count and a cheaper overall cost for the task, proving that cheaper-per-token isn't always cheaper-per-task.

Focusing on token pricing is misleading. A more powerful model may be more expensive per token but significantly cheaper per task because its higher efficiency requires fewer prompts and iterations to achieve a final result. The correct way to measure cost-effectiveness is by the total cost to complete a job, not the price of the raw material.