The performance gains from Nvidia's Hopper to Blackwell GPUs come from increased size and power, not efficiency. This signals a potential scaling limit, creating an opportunity for radically new hardware primitives and neural network architectures beyond today's matrix-multiplication-centric models.

Jensen Huang emphasizes that Moore's Law is dead as a primary performance driver. The 50x gain from Hopper to Blackwell came overwhelmingly from architecture and computer science breakthroughs, with raw transistor improvements providing only marginal benefit.

Nvidia’s advantage over ASICs like Google's TPU is programmability. While ASICs are limited to Moore's Law's slow annual gains, CUDA enables radical algorithmic changes that create 10-100x performance leaps, as seen in the jump from Hopper to Blackwell.

Huawei is shifting from shrinking transistors (Moore's Law) to optimizing data flow via advanced chip stacking and interconnects. This "tau scaling law" is an innovative workaround to physical limits, aiming to create competitive AI compute power without access to the most advanced manufacturing processes.

The next wave of AI silicon may pivot from today's compute-heavy architectures to memory-centric ones optimized for inference. This fundamental shift would allow high-performance chips to be produced on older, more accessible 7-14nm manufacturing nodes, disrupting the current dependency on cutting-edge fabs.

The core architectural bet for Cerebras was that incremental improvements on an existing design (like a GPU) yield minimal gains because the incumbent has already optimized it. To achieve a step-change in performance, a fundamentally different approach is required, leading them to their massive, wafer-scale chip design.

A key trend in AI models is "dynamism"—the ability to vary computation and memory usage per token, as seen in Mixture-of-Experts (MoE) architectures. Current hardware, designed before this trend, is inefficient. New chips must be built to accelerate these dynamic computations.

Andrew Feldman, CEO of competitor Cerebras, argues their single wafer-scale chip is superior for large AI models. He contends that connecting thousands of smaller GPUs, as Nvidia does, introduces significant latency from physical wiring that negates on-paper performance specs, creating a fundamental bottleneck.

Unlike GPUs using slow, dense memory, Cerebras's wafer-sized chip leverages its vast surface area to accommodate faster, less-dense memory. This design sidesteps memory bottlenecks, achieving speeds up to 15 times faster than the fastest GPUs for AI tasks.