Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Microsoft Azure imposes a harsh "use-it-or-lose-it" policy on GPU clusters for smaller customers. Even a few hours of underutilization can result in being kicked off and placed at the back of a months-long waiting list, creating major instability for startups.

Related Insights

When major infrastructure like AWS or Cloudflare goes down, it affects many companies simultaneously. This creates a collective "mulligan," meaning individual startups aren't heavily penalized by users for the downtime, as the issue is widespread. The exception is for mission-critical services like finance or live events.

At scale, renting compute from AWS, Google, or Microsoft is a strategic mistake for AI leaders like OpenAI and Anthropic. It creates a critical dependency, forcing them to enter the capital-intensive data center business to control their supply chain and destiny.

Amidst a 48% spike in GPU rental costs, AI companies like Anthropic are shifting heavy enterprise users from flat-rate to usage-based pricing. This move, framed as unblocking power users, is fundamentally a response to the industry-wide compute shortage, directly linking the high cost-to-serve with customer pricing.

Anthropic is throttling user access during peak hours due to GPU shortages. This confirms that the AI industry remains severely compute-constrained and validates the multi-billion dollar infrastructure investments by giants like OpenAI and Meta, which once seemed excessive.

Large tech companies are buying up compute from smaller cloud providers not for immediate need, but as a defensive strategy. By hoarding scarce GPU capacity, they prevent competitors from accessing critical resources, effectively cornering the market and stifling innovation from rivals.

AI labs like Anthropic that were conservative in securing long-term compute now face a 'quality tax.' They must resort to lower-quality providers or pay significant markups and revenue-sharing deals for last-minute capacity, a cost their more aggressive competitors like OpenAI avoided by signing deals early.

Once a haven for startups struggling to get GPUs, NeoClouds like CoreWeave have shifted their strategy. They now prioritize serving the largest customers, mirroring the behavior of AWS and Azure and leaving startups with fewer alternative compute options than in 2023.

Satya Nadella reveals that Microsoft prioritizes building a flexible, "fungible" cloud infrastructure over catering to every demand of its largest AI customer, OpenAI. This involves strategically denying requests for massive, dedicated data centers to ensure capacity remains balanced for other customers and Microsoft's own high-margin products.

Despite strong earnings and its OpenAI partnership, Microsoft's stock dropped because limited AI hardware and data center capacity are constraining Azure's revenue growth. This shows physical infrastructure is a major bottleneck for cloud giants, directly impacting market perception.

A speaker theorizes that increased cloud outages are not random. Cloud providers, rushing to buy GPUs for AI, have underinvested in refreshing their general-purpose CPU infrastructure. With CPUs now hitting their 5-year end-of-life and new AI-related CPU demand rising, the system is becoming strained and unstable.