The key measure of leverage for AI-powered developers is no longer GPU utilization (FLOPs) but the volume of tokens processed by agents. Karpathy feels nervous when his token subscriptions are underutilized, indicating he's the bottleneck, not the system.

Intense demand for AI tokens is outstripping compute supply, making flat-rate SaaS pricing unsustainable. Companies like GitHub are now shifting to usage-based billing to cover escalating inference costs, marking a fundamental change in how AI products are sold and signaling a broader industry trend.

For companies at the trillion-token scale, cost predictability is more important than the lowest per-token price. Superhuman favors providers offering fixed-capacity pricing, giving them better control over their cost structure, which is crucial for pre-IPO financial planning.

The advertised per-hour GPU cost is misleading. Because research workloads are spiky and unpredictable, labs over-provision compute. This rampant underutilization means the effective price paid is often 10 times higher than the marketed rate, creating massive deadweight loss.

The GPU architecture is economically optimized for slow AI inference, offering a very low cost per token. However, this efficiency plummets when speed is required, as the cost and power per token increase exponentially, creating a market for alternative architectures in high-speed applications.

The high cost of GPUs means any inefficiency during model training is extremely expensive. This economic reality justifies building specialized, AI-focused infrastructure with features like advanced observability and optimized storage to maximize GPU utilization and prevent costly delays from failures or slowdowns.

While the growth of new consumer AI users is slowing into an S-curve, the compute consumption per user is still growing exponentially. This is driven by the shift from simple queries to complex, token-intensive tasks like reasoning and agents, sustaining massive demand for GPU infrastructure.

Despite enterprises hitting AI budget limits, the market is not collapsing. Competition is forcing AI providers to lower token prices, triggering the Jevons paradox: as a resource's cost falls, its consumption increases, sustaining demand for underlying infrastructure like NVIDIA chips.

The high operational cost of using proprietary LLMs creates 'token junkies' who burn through cash rapidly. This intense cost pressure is a primary driver for power users to adopt cheaper, local, open-source models they can run on their own hardware, creating a distinct market segment.