Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Goldman's CIO predicts that while unit cost per token will decrease, the explosion in token usage from agentic systems will make total AI compute a major corporate expense. He suggests it should be compared to personnel costs, not traditional IT spending.

Related Insights

The team managing Composio's AI pipeline for building tool integrations spends more on LLM tokens than on salaries for its engineers. This signals a new economic reality for AI-native companies where compute is a larger operational cost than labor.

Historically, payroll has dominated corporate expenses. As AI automates knowledge work previously done by humans, a significant portion of the budget will shift. Spend on SaaS, APIs, and model usage will grow from a small percentage to a major line item, displacing traditional labor costs.

Historically, labor costs dwarfed software spending. As AI automates tasks, software budgets will balloon, turning into a primary corporate expense. This forces CFOs to scrutinize software ROI with the same rigor they once applied only to their workforce.

Ramp's CPO argues companies shouldn't excessively worry about AI token costs. If an AI agent can deliver 10x the output of a human, it's logical and profitable to pay the agent (via tokens) more than the human's salary. This reframes ROI from a cost center to a massive productivity investment.

A paradox exists where the cost for a fixed level of AI capability (e.g., GPT-4 level) has dropped 100-1000x. However, overall enterprise spend is increasing because applications now use frontier models with massive contexts and multi-step agentic workflows, creating huge multipliers on token usage that drive up total costs.

While the per-unit cost of using AI has plummeted, total enterprise spending has soared. This is a classic example of the Jevons paradox: efficiency gains and lower prices are unlocking entirely new use cases that were previously uneconomical, leading to a net increase in overall consumption and total expenditure.

The cost of AI, priced in "tokens by the drink," is falling dramatically. All inputs are on a downward cost curve, leading to a hyper-deflationary effect on the price of intelligence. This, in turn, fuels massive demand elasticity as more use cases become economically viable.

While the cost to achieve a fixed capability level (e.g., GPT-4 at launch) has dropped over 100x, overall enterprise spending is increasing. This paradox is explained by powerful multipliers: demand for frontier models, longer reasoning chains, and multi-step agentic workflows that consume exponentially more tokens.

Heavy use of AI agents and API calls is generating significant costs, with some agents costing $100,000 annually. This creates a new financial reality where companies must budget for 'tokens' per employee, potentially making the AI's cost more than the human's salary.

While the cost for GPT-4 level intelligence has dropped over 100x, total enterprise AI spend is rising. This is driven by multipliers: using larger frontier models for harder tasks, reasoning-heavy workflows that consume more tokens, and complex, multi-turn agentic systems.

Per-Token Costs Will Drop, But Total AI Spend Will Become a Major Expense Line | RiffOn