Andreessen asserts that the AI models we use daily are intentionally limited versions of what labs have developed. The primary constraint is not research progress but the severe shortage of GPU capacity. If compute were plentiful, current models would be significantly more powerful.

While focus is on massive supercomputers for training next-gen models, the real supply chain constraint will be 'inference' chips—the GPUs needed to run models for billions of users. As adoption goes mainstream, demand for everyday AI use will far outstrip the supply of available hardware.

Templar's Sam Dare argues the perceived GPU scarcity is misunderstood. The actual bottleneck is the limited supply of the latest, well-connected GPUs in data centers. His project aims to create algorithms that can effectively utilize the vast, distributed network of consumer-grade and older enterprise GPUs, unlocking a massive new compute resource.

Anthropic is throttling user access during peak hours due to GPU shortages. This confirms that the AI industry remains severely compute-constrained and validates the multi-billion dollar infrastructure investments by giants like OpenAI and Meta, which once seemed excessive.

While NVIDIA's GPUs have been the primary AI constraint, the bottleneck is now moving to other essential subsystems. Memory, networking interconnects, and power management are emerging as the next critical choke points, signaling a new wave of investment opportunities in the hardware stack beyond core compute.

A critical, under-discussed constraint on Chinese AI progress is the compute bottleneck caused by inference. Their massive user base consumes available GPU capacity serving requests, leaving little compute for the R&D and training needed to innovate and improve their models.

While many focus on compute metrics like FLOPS, the primary bottleneck for large AI models is memory bandwidth—the speed of loading weights into the GPU. This single metric is a better indicator of real-world performance from one GPU generation to the next than raw compute power.

While the world focused on GPU shortages, the real constraint on AI compute is now physical infrastructure. The bottleneck has moved to accessing power, building data centers, and finding specialized labor like electricians and acquiring basic materials like structural steel. Merely acquiring chips is no longer enough to scale.

Efficiency gains in new chips like NVIDIA's H200 don't lower overall energy use. Instead, developers leverage the added performance to build larger, more complex models. This "ambition creep" negates chip-level savings by increasing training times and data movement, ultimately driving total system power consumption higher.