The physical area a multiplier circuit requires on a chip grows quadratically with the number of bits (e.g., p*q). This non-linear scaling is the fundamental reason why lower-precision formats like FP4 and FP8 offer disproportionately large performance and efficiency gains for AI workloads compared to a linear improvement.

Since the weight matrix in a systolic array is reused many times, it doesn't need to be loaded quickly. Chip designers can use slow, low-bandwidth connections to "trickle feed" the weights, minimizing the required wiring and thus saving precious die area. This prioritizes area efficiency over initial load latency.

The next wave of AI silicon may pivot from today's compute-heavy architectures to memory-centric ones optimized for inference. This fundamental shift would allow high-performance chips to be produced on older, more accessible 7-14nm manufacturing nodes, disrupting the current dependency on cutting-edge fabs.

The necessity of batching stems from a fundamental hardware reality: moving data is far more energy-intensive than computing with it. A single parameter's journey from on-chip SRAM to the multiplier can cost 1000x more energy than the multiplication itself. Batching amortizes this high data movement cost over many computations.

While you can insert registers (pipelining) to shorten simple logic paths and increase clock speed, you cannot easily do this with a feedback loop (e.g., an accumulator). The time it takes for a signal to traverse this recurring loop becomes the fundamental constraint that dictates the entire chip's maximum clock frequency.

While adding pipeline registers can increase a chip's clock speed, the registers themselves consume significant silicon area. Over-pipelining can lead to a chip where most of the area is dedicated to registers, not useful logic, resulting in lower overall throughput despite the high clock frequency.

An FPGA's inefficiency stems from its programmable nature. A simple 3-gate 'AND' circuit in a custom ASIC is implemented on an FPGA using a generic lookup table (LUT). This LUT, which is essentially a multiplexer, might require over 30 gates to build, creating a ~10x overhead in area and power.

Systolic arrays (like NVIDIA's Tensor Cores) overcome the high cost of data movement by storing the large, reusable weight matrix directly within the compute fabric. This avoids repeatedly fetching weights from a distant register file, dramatically improving the ratio of computation to communication.

Cerebras's innovative wafer-scale architecture has a major flaw: on-chip SRAM memory is not scaling with new semiconductor nodes. This creates a difficult trade-off between compute and memory, limiting the chip's ability to handle increasingly larger AI models and context windows, as shown by the mere 10% memory increase in its latest chip.