Arvind Krishna forecasts a 1000x drop in AI compute costs over five years. This won't just come from better chips (a 10x gain). It will be compounded by new processor architectures (another 10x) and major software optimizations like model compression and quantization (a final 10x).
The AI race has been a prisoner's dilemma where companies spend massively, fearing competitors will pull ahead. As the cost of next-gen systems like Blackwell and Rubin becomes astronomical, the sheer economics will force a shift. Decision-making will be dominated by ROI calculations rather than the existential dread of slowing down.
The progression from early neural networks to today's massive models is fundamentally driven by the exponential increase in available computational power, from the initial move to GPUs to today's million-fold increases in training capacity on a single model.
The AI era is not an unprecedented bubble but the next phase in a recurring pattern where each new computing cycle (mainframe, PC, internet) is roughly 10 times larger than the last. This historical context suggests the current massive investment is proportional and we are still in the early innings.
When power (watts) is the primary constraint for data centers, the total cost of compute becomes secondary. The crucial metric is performance-per-watt. This gives a massive pricing advantage to the most efficient chipmakers, as customers will pay anything for hardware that maximizes output from their limited power budget.
Today's AI is largely text-based (LLMs). The next phase involves Visual Language Models (VLMs) that interpret and interact with the physical world for robotics and surgery. This transition requires an exponential, 50-1000x increase in compute power, underwriting the long-term AI infrastructure build-out.
AI progress was expected to stall in 2024-2025 due to hardware limitations on pre-training scaling laws. However, breakthroughs in post-training techniques like reasoning and test-time compute provided a new vector for improvement, bridging the gap until next-generation chips like NVIDIA's Blackwell arrived.
Chinese AI models like Kimi achieve dramatic cost reductions through specific architectural choices, not just scale. Using a "mixture of experts" design, they only utilize a fraction of their total parameters for any given task, making them far more efficient to run than the "dense" models common in the West.
AI's computational needs are not just from initial training. They compound exponentially due to post-training (reinforcement learning) and inference (multi-step reasoning), creating a much larger demand profile than previously understood and driving a billion-X increase in compute.
The infrastructure demands of AI have caused an exponential increase in data center scale. Two years ago, a 1-megawatt facility was considered a good size. Today, a large AI data center is a 1-gigawatt facility—a 1000-fold increase. This rapid escalation underscores the immense and expensive capital investment required to power AI.
Responding to the AI bubble concern, IBM's CEO notes high GPU failure rates are a design choice for performance. Unlike sunken costs from past bubbles, these "stranded" hardware assets can be detuned to run at lower power, increasing their resilience and extending their useful life for other tasks.