Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Advice from AI frontier labs often presumes endless compute and token subsidies, which is impractical for production software engineers operating within real budgets. Michael warns developers to critically evaluate lab recommendations and explore open-weight models to avoid vendor dependency.

Related Insights

AI labs profit from token generation, creating a "big token" economy that conflicts with enterprise budgets. The solution is to use a portfolio of models—large ones for complex tasks and smaller, cheaper ones for simple edits—to optimize the cost-performance ratio.

Developers are increasingly using expensive frontier models only for high-stakes tasks, flocking to cheaper, open-weight alternatives for everything else. This price pressure is commoditizing AI, which could implode the debt-fueled valuations of leading AI companies.

With frontier models costing over 100x more than competent alternatives ($56 vs. 50¢ per million tokens), companies are burning cash. An estimated 98% of tasks sent to top-tier models don't require that power, an inefficiency driven by engineers who are disconnected from cost implications.

As enterprise usage ramps up, LLM token bills threaten corporate budgets, creating intense pressure on CIOs and CFOs to reduce costs. Enterprises do not need frontier-grade intelligence for every workflow; near-frontier open-weight models that trail frontier models by 6 to 12 months can handle 90% of reasoning, coding, and agentic tasks at a third of the cost. This dynamic will force enterprises to downgrade non-critical workloads away from expensive closed models.

State-of-the-art flagship models often overcomplicate basic tasks and consume excessive tokens compared to efficient smaller models. Engineers should reserve top-tier models for orchestration and code review while delegating routine knowledge or computation tasks to cheaper alternatives.

Frontier model providers like OpenAI are not incentivized to reduce your token costs, creating lock-in. Independent agent labs (e.g., Cognition) act like brokers, routing tasks to the most efficient models, which is a more sustainable and affordable long-term strategy for building your company.

The biggest AI labs promote their frontier models, but these are often unnecessary for real-world enterprise agentic workflows. More practical and cost-effective solutions can be achieved using smaller proprietary models (like Anthropic's Sonnet) or even open-source alternatives like Muse.

The most advanced AI models are not universally superior; their capabilities form a "jagged frontier." This means organizations can often use more economical, locally-run open-weight models for tasks where they are "good enough," reserving expensive frontier models for specialized needs.

As AI token consumption becomes a major budget item, companies are moving beyond using a single frontier model. Every organization will need a portfolio of models, including cheaper options for less complex tasks, to manage the "madness" of runaway costs.

Even in competitive fields like hedge funds, using the most expensive frontier AI models is not a clear win. The cost of "token maxing" is untethered from revenue, meaning a business can easily become unprofitable by chasing the latest model without being able to pass those costs on.

Frontier Lab Best Practices Rely on Unrealistic Unlimited Token Budgets | RiffOn