A 10x increase in compute may only yield a one-tier improvement in model performance. This appears inefficient but can be the difference between a useless "6-year-old" intelligence and a highly valuable "16-year-old" intelligence, unlocking entirely new economic applications.

The performance gains from Nvidia's Hopper to Blackwell GPUs come from increased size and power, not efficiency. This signals a potential scaling limit, creating an opportunity for radically new hardware primitives and neural network architectures beyond today's matrix-multiplication-centric models.

Since the weight matrix in a systolic array is reused many times, it doesn't need to be loaded quickly. Chip designers can use slow, low-bandwidth connections to "trickle feed" the weights, minimizing the required wiring and thus saving precious die area. This prioritizes area efficiency over initial load latency.

AI model capabilities follow a predictable, non-linear scaling law: increasing training compute by 10x roughly doubles a model's capabilities. This exponential relationship, rather than an incremental one, is what will drive underappreciated and disruptive advancements across many industries.

The necessity of batching stems from a fundamental hardware reality: moving data is far more energy-intensive than computing with it. A single parameter's journey from on-chip SRAM to the multiplier can cost 1000x more energy than the multiplication itself. Batching amortizes this high data movement cost over many computations.

The relationship between computing power and AI model capability is not linear. According to established 'scaling laws,' a tenfold increase in the compute used for training large language models (LLMs) results in roughly a doubling of the model's capabilities, highlighting the immense resources required for incremental progress.

The fundamental primitive for AI chips isn't arbitrary; it's the multiply-accumulate (MAC) operation. This is because it directly maps to the innermost computational loop of matrix multiplication (output += input1 * input2), which is the foundational computation for most neural networks.

Current AI models become exponentially more expensive as input size grows (quadratic scaling). New "subquadratic" architectures, however, scale linearly by pre-selecting relevant data. This change could slash compute costs by orders of magnitude, making massive context windows economically viable.

Modern AI models are moving towards extremely low-precision arithmetic (e.g., 4-bit numbers) because it's more efficient. The trade-off is analogous to image processing: you get a better result with more pixels (more computations) and fewer colors (less precision) than the other way around.