Despite the hype around large language models, they represent a minority of AI compute usage at a tech giant like Meta. The vast majority of AI capital expenditure is dedicated to other tasks like content recommendation and ad placement, highlighting the continued importance of diverse, non-LLM AI systems in large-scale operations.

It's counterintuitive, but using a more expensive, intelligent model like Opus 4.5 can be cheaper than smaller models. Because the smarter model is more efficient and requires fewer interactions to solve a problem, it ends up using fewer tokens overall, offsetting its higher per-token price.

The new multi-agent architecture in Opus 4.6, while powerful, dramatically increases token consumption. Each agent runs its own process, multiplying token usage for a single prompt. This is a savvy business strategy, as the model's most advanced feature is also its most lucrative for Anthropic.

Meta's massive internal consumption of AI tokens for tasks like code generation creates a multi-billion dollar expense. By developing its own frontier models in-house, Meta can vertically integrate, justifying the high cost of its AI lab (MSL) purely on internal savings, even before launching any new consumer AI products.

A practical hack to combat rising AI API costs is instructing models to respond with minimal, non-grammatical language. By using prompts like "did thing" instead of a full sentence, users can drastically reduce token consumption for a given task, directly lowering operational expenses.

The $15-$25 per-review price for Anthropic's tool moves AI expenses from a predictable monthly software subscription to a variable cost that scales like human labor. This forces CTOs to justify AI budgets with direct headcount savings, creating immense pressure on ROI.

A paradox exists where the cost for a fixed level of AI capability (e.g., GPT-4 level) has dropped 100-1000x. However, overall enterprise spend is increasing because applications now use frontier models with massive contexts and multi-step agentic workflows, creating huge multipliers on token usage that drive up total costs.

While the cost to achieve a fixed capability level (e.g., GPT-4 at launch) has dropped over 100x, overall enterprise spending is increasing. This paradox is explained by powerful multipliers: demand for frontier models, longer reasoning chains, and multi-step agentic workflows that consume exponentially more tokens.

Goldman's CIO predicts that while unit cost per token will decrease, the explosion in token usage from agentic systems will make total AI compute a major corporate expense. He suggests it should be compared to personnel costs, not traditional IT spending.