The AI hardware market is fragmenting. Google is now producing two distinct eighth-generation TPUs: one for training (8t) and one for inference (8i). This move away from one-size-fits-all GPUs shows that optimizing for specific AI workloads is the next competitive frontier.

Google isn't betting on a single chip design. It's actively developing three distinct TPU architectures with different partners to avoid being trapped in a "local minima." This hedges against future breakthroughs in model architecture that could render one design obsolete.

Google is abandoning its single-line TPU strategy, now working with both Broadcom and MediaTek on different, specialized TPU designs. This reflects an industry-wide realization that no single chip can be optimal for the diverse and rapidly evolving landscape of AI tasks.

Designing custom AI hardware is a long-term bet. Google's TPU team co-designs chips with ML researchers to anticipate future needs. They aim to build hardware for the models that will be prominent 2-6 years from now, sometimes embedding speculative features that could provide massive speedups if research trends evolve as predicted.

Instead of running an entire inference task on a single GPU, the next efficiency leap will come from breaking it down. Tasks like prefill, attention, and feed-forward networks will be routed to specialized chips, such as SRAM-based accelerators, that are best suited for each job, dramatically improving performance and ROI.

The fear of chip depreciation is mitigated by repurposing older GPUs. While the latest models are crucial for high-speed training, older, less powerful chips are perfectly suitable and cost-effective for running inference, extending the hardware's useful life and long-term value.

Anthropic's strategy of running workloads on diverse chips (NVIDIA, Google TPU, AWS Trainium) is less about long-term diversification and more about immediate survival. In a market where compute is severely constrained, the ability to utilize any available chip becomes a critical competitive advantage, forcing deep technical competence across architectures.

Contrary to fears of rapid obsolescence, new domain-specific accelerators (DSAs) can be paired with older GPUs to handle specific tasks. This disaggregated approach extends the useful life of GPUs to 10-15 years, lowering financing costs for compute providers and invalidating bear cases.

Anthropic mitigates supply chain risk and optimizes cost by investing heavily in the ability to use NVIDIA, Google, and Amazon chips interchangeably for model development, internal use, and customer service. This orchestration layer is a key competitive advantage.