Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Evan Conrad estimates a B300 can make about $40/hour selling inference at full utilization, or $25/hour at 50%, while B300s rent for $5-6/hour. He says one of those numbers must change, calling it a huge arbitrage. That gap shapes willingness to pay for future capacity.

Related Insights

Firms like OpenAI and Meta claim a compute shortage while also exploring selling compute capacity. This isn't a contradiction but a strategic evolution. They are buying all available supply to secure their own needs and then arbitraging the excess, effectively becoming smaller-scale cloud providers for AI.

Contrary to the narrative of burning cash, major AI labs are likely highly profitable on the marginal cost of inference. Their massive reported losses stem from huge capital expenditures on training runs and R&D. This financial structure is more akin to an industrial manufacturer than a traditional software company, with high upfront costs and profitable unit economics.

The advertised per-hour GPU cost is misleading. Because research workloads are spiky and unpredictable, labs over-provision compute. This rampant underutilization means the effective price paid is often 10 times higher than the marketed rate, creating massive deadweight loss.

While AI compute demand seems limitless, its price is not infinitely elastic. As inference becomes a core cost of goods sold (COGS) for AI products, excessively high compute prices will break the business models of infrastructure customers, ultimately limiting demand.

The perceived constraint on AI compute isn't a true supply issue, but a consequence of VC-funded companies pricing their services below cost to fuel growth. This creates artificial demand that masks the true, profitable market size until unit economics are forced.

As AI models achieve human-level capabilities in valuable roles like software engineering, they can generate significantly more revenue from the same hardware. This increased monetization potential will cause the rental price of GPUs to skyrocket, potentially by over 15x, to match the economic value they produce.

While a megawatt of compute costs ~$15M, leading labs like Anthropic can generate up to $50M in revenue from it. This massive 3x+ profit margin creates a powerful flywheel, allowing them to reinvest heavily in training the next generation of models and accelerate their lead.

Instead of viewing compute as a cost center, OpenAI treats it as a revenue generator, analogous to hiring salespeople. The core belief is that demand for AI capabilities is so vast that they can never build compute fast enough to satisfy it, justifying massive, forward-looking infrastructure investments.

Drawing a parallel to AWS's history, AI inference costs are expected to continuously decrease over time. As usage skyrockets, providers will be incentivized to lower prices to capture market share, making fears of escalating costs for startups unlikely to materialize.

The market fears rising credit costs will stall the AI buildout. However, existing GPU compute is contracted at prices far below current spot rates. As these contracts expire, repricing will accelerate hyperscaler operating cash flow, allowing them to self-fund expansion without needing as much debt.