Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Evan Conrad describes the order book as a market for breaking leases. Batch inference can accept interruption in minutes; live inference customers may need months of notice and face SLA penalties. Those interruption rights price differently, similar to power markets' day-ahead and live markets. The terms are the product.

Related Insights

Amidst a 48% spike in GPU rental costs, AI companies like Anthropic are shifting heavy enterprise users from flat-rate to usage-based pricing. This move, framed as unblocking power users, is fundamentally a response to the industry-wide compute shortage, directly linking the high cost-to-serve with customer pricing.

AI applications often have long waiting periods for model responses or user input, but traditional cloud platforms charge for this idle time. Vercel's "Fluid Compute" is designed so customers only pay when the application is actively processing, making it fundamentally more cost-effective for AI workloads.

Long-term capacity deals can backfire for GPU clouds during periods of rising compute prices. Dylan Patel points out that CoreWeave and Oracle signed massive AI contracts at fixed margins, preventing them from capturing price upside as GPU costs increased. In contrast, Nebius maintained shorter average contract durations, allowing them to raise rental rates and capitalize on intense market demand far more aggressively.

Modern software products increasingly incur variable compute and inference costs for research and product delivery, making static subscriptions economically fragile. John Collison explains that nearly all AI products must introduce usage-based components, credit systems, or overage charges to align consumer pricing directly with underlying model expenses.

While AI compute demand seems limitless, its price is not infinitely elastic. As inference becomes a core cost of goods sold (COGS) for AI products, excessively high compute prices will break the business models of infrastructure customers, ultimately limiting demand.

Evan Conrad estimates a B300 can make about $40/hour selling inference at full utilization, or $25/hour at 50%, while B300s rent for $5-6/hour. He says one of those numbers must change, calling it a huge arbitrage. That gap shapes willingness to pay for future capacity.

Inference costs fluctuate based on regional electricity rates and idle GPU capacity across distributed providers. Smart order routing architectures that run prompt-level mini-auctions can undercut static aggregators by allowing compute providers to continuously adjust prices based on marginal cost.

Evan Conrad says two VMs from one operator can differ by 40% in kernel-launch performance, and storage, WAN transit, preemption and chip generation also change value. Rather than wait for standardized contracts, SF Compute lets buyers and sellers attach multiple properties to orders, accepting wider spreads and arbitrage for flexible buyers.

With SaaS, a lack of value might not be exposed until renewal. With consumption, customers can "turn the light switch off" instantly, forcing vendors to prove their worth continuously and re-earn the business every day.

Startups training foundation models face a new existential threat: the death of on-demand compute. Cloud providers, leveraging scarcity, now push for expensive three-to-five-year contracts. This forces early-stage companies into massive, long-term commitments they can ill afford and whose future needs are highly uncertain.