While focus is on massive supercomputers for training next-gen models, the real supply chain constraint will be 'inference' chips—the GPUs needed to run models for billions of users. As adoption goes mainstream, demand for everyday AI use will far outstrip the supply of available hardware.

While AI inference can be decentralized, training the most powerful models demands extreme centralization of compute. The necessity for high-bandwidth, low-latency communication between GPUs means the best models are trained by concentrating hardware in the smallest possible physical space, a direct contradiction to decentralized ideals.

Simply "scaling up" (adding more GPUs to one model instance) hits a performance ceiling due to hardware and algorithmic limits. True large-scale inference requires "scaling out" (duplicating instances), creating a new systems problem of managing and optimizing across a distributed fleet.

The current focus on building massive, centralized AI training clusters represents the 'mainframe' era of AI. The next three years will see a shift toward a distributed model, similar to computing's move from mainframes to PCs. This involves pushing smaller, efficient inference models out to a wide array of devices.

While AI training requires massive, centralized data centers, the growth of inference workloads is creating a need for a new architecture. This involves smaller (e.g., 5 megawatt), decentralized clusters located closer to users to reduce latency. This shift impacts everything from data center design to the software required to manage these distributed fleets.

The era of dual-purpose AI chips is ending. The overwhelming demand for real-time processing from AI agents is forcing companies like Google and NVIDIA to create dedicated, inference-optimized hardware. This marks a fundamental and permanent split in the AI infrastructure market, separating training from inference.

Previously, the biggest constraint in AI was compute for training next-gen models. Now, the critical bottleneck is providing enough compute for *inference*—the real-time processing of queries from a rapidly growing user base.

When splitting jobs across thousands of GPUs, inconsistent communication times (jitter) create bottlenecks, forcing the use of fewer GPUs. A network with predictable, uniform latency enables far greater parallelization and overall cluster efficiency, making it more important than raw 'hero number' bandwidth.

The race for compute power is moving from centralized data centers to decentralized networks. Companies are already putting GPU clusters next to homes, and Tesla is positioned to leverage its Powerwalls and Starlink for a distributed compute system that bypasses traditional infrastructure bottlenecks.