A 10x increase in compute may only yield a one-tier improvement in model performance. This appears inefficient but can be the difference between a useless "6-year-old" intelligence and a highly valuable "16-year-old" intelligence, unlocking entirely new economic applications.

The original playbook of simply scaling parameters and data is now obsolete. Top AI labs have pivoted to heavily designed post-training pipelines, retrieval, tool use, and agent training, acknowledging that raw scaling is insufficient to solve real-world problems.

The relationship between computing power and AI model capability is not linear. According to established 'scaling laws,' a tenfold increase in the compute used for training large language models (LLMs) results in roughly a doubling of the model's capabilities, highlighting the immense resources required for incremental progress.

Over two-thirds of reasoning models' performance gains came from massively increasing their 'thinking time' (inference scaling). This was a one-time jump from a zero baseline. Further gains are prohibitively expensive due to compute limitations, meaning this is not a repeatable source of progress.

The plateauing performance-per-watt of GPUs suggests that simply scaling current matrix multiplication-heavy architectures is unsustainable. This hardware limitation may necessitate research into new computational primitives and neural network designs built for large-scale distributed systems, not single devices.

While AI inference can be decentralized, training the most powerful models demands extreme centralization of compute. The necessity for high-bandwidth, low-latency communication between GPUs means the best models are trained by concentrating hardware in the smallest possible physical space, a direct contradiction to decentralized ideals.

The focus in AI engineering is shifting from making a single agent faster (latency) to running many agents in parallel (throughput). This "wider pipe" approach gets more total work done but will stress-test existing infrastructure like CI/CD, which wasn't built for this volume.

The current focus on building massive, centralized AI training clusters represents the 'mainframe' era of AI. The next three years will see a shift toward a distributed model, similar to computing's move from mainframes to PCs. This involves pushing smaller, efficient inference models out to a wide array of devices.

While AI training requires massive, centralized data centers, the growth of inference workloads is creating a need for a new architecture. This involves smaller (e.g., 5 megawatt), decentralized clusters located closer to users to reduce latency. This shift impacts everything from data center design to the software required to manage these distributed fleets.