As chip manufacturers like NVIDIA release new hardware, inference providers like Base10 absorb the complexity and engineering effort required to optimize AI models for the new chips. This service is a key value proposition, saving customers from the challenging process of re-optimizing workloads for new hardware.
Related Insights
The Rubin family of chips is sold as a complete "system as a rack," meaning customers can't just swap out old GPUs. This technical requirement creates a forced, expensive upgrade cycle for cloud providers, compelling them to invest heavily in entirely new rack systems to stay competitive.
New AI models are designed to perform well on available, dominant hardware like NVIDIA's GPUs. This creates a self-reinforcing cycle where the incumbent hardware dictates which model architectures succeed, making it difficult for superior but incompatible chip designs to gain traction.
While NVIDIA's CUDA software provides a powerful lock-in for AI training, its advantage is much weaker in the rapidly growing inference market. New platforms are demonstrating that developers can and will adopt alternative software stacks for deployment, challenging the notion of an insurmountable software moat.
NVIDIA's complex Blackwell chip transition requires rapid, large-scale deployment to work out bugs. XAI, known for building data centers faster than anyone, serves this role for NVIDIA. This symbiotic relationship helps NVIDIA stabilize its new platform while giving XAI first access to next-generation models.
![Gavin Baker - Nvidia v. Google, Scaling Laws, and the Economics of AI - [Invest Like the Best, EP.451] thumbnail](https://megaphone.imgix.net/podcasts/d97fee14-d4d7-11f0-8951-8324b640c1c1/image/d3a8b5ecbf3957de5b91f278a191c9ff.jpg?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress)
The massive investment in data centers isn't just a bet on today's models. As AI becomes more efficient, smaller yet powerful models will be deployed on older hardware. This extends the serviceable life and economic return of current infrastructure, ensuring today's data centers will still generate value years from now.
Model architecture decisions directly impact inference performance. AI company Zyphra pre-selects target hardware and then chooses model parameters—such as a hidden dimension with many powers of two—to align with how GPUs split up workloads, maximizing efficiency from day one.
NVIDIA’s business model relies on planned obsolescence. Its AI chips become obsolete every 2-3 years as new versions are released, forcing Big Tech customers into a constant, multi-billion dollar upgrade cycle for what are effectively "perishable" assets.
Nvidia will likely only revive its ambitions to compete with AWS if its massive hardware profit margins are threatened by competitors like AMD or hyperscalers building their own chips. Only then would Nvidia move up the stack to capture value through an "inference as a service" business model, moving beyond hardware sales.
While Nvidia dominates the AI training chip market, this only represents about 1% of the total compute workload. The other 99% is inference. Nvidia's risk is that competitors and customers' in-house chips will create cheaper, more efficient inference solutions, bifurcating the market and eroding its monopoly.
OpenAI is designing its custom chip for flexibility, not just raw performance on current models. The team learned that major 100x efficiency gains come from evolving algorithms (e.g., dense to sparse transformers), so the hardware must be adaptable to these future architectural changes.
