While focus is on massive supercomputers for training next-gen models, the real supply chain constraint will be 'inference' chips—the GPUs needed to run models for billions of users. As adoption goes mainstream, demand for everyday AI use will far outstrip the supply of available hardware.

The focus in AI has evolved from rapid software capability gains to the physical constraints of its adoption. The demand for compute power is expected to significantly outstrip supply, making infrastructure—not algorithms—the defining bottleneck for future growth.

While NVIDIA's GPUs have been the primary AI constraint, the bottleneck is now moving to other essential subsystems. Memory, networking interconnects, and power management are emerging as the next critical choke points, signaling a new wave of investment opportunities in the hardware stack beyond core compute.

The critical constraint on AI and future computing is not energy consumption but access to leading-edge semiconductor fabrication capacity. With data centers already consuming over 50% of advanced fab output, consumer hardware like gaming PCs will be priced out, accelerating a fundamental shift where personal devices become mere terminals for cloud-based workloads.

The focus on GPUs for AI overlooks a critical bottleneck: CPU shortages. AI agents require massive CPU power for non-GPU tasks like web queries and data prep. This demand is straining existing infrastructure and creating new market opportunities for CPU makers like ARM.

While the world focused on GPU shortages, the real constraint on AI compute is now physical infrastructure. The bottleneck has moved to accessing power, building data centers, and finding specialized labor like electricians and acquiring basic materials like structural steel. Merely acquiring chips is no longer enough to scale.

While the growth of new consumer AI users is slowing into an S-curve, the compute consumption per user is still growing exponentially. This is driven by the shift from simple queries to complex, token-intensive tasks like reasoning and agents, sustaining massive demand for GPU infrastructure.

The rise of agent orchestration using specialized, open-source models will drive demand for custom ASICs. Jerry Murdock argues that putting a model on a dedicated chip will be far cheaper and more tunable for specific workloads than using expensive, general-purpose GPUs like Nvidia's, spurring a hardware shift.

After the current memory crunch, the next AI infrastructure bottleneck will be CPU and networking. The complex orchestration required for emerging agentic AI systems will strain these resources, a trend already visible in companies like Fastly seeing demand spikes just for workload orchestration.