Breakthroughs like neural network "pruning" can reduce model size by 90% without losing accuracy, offering a 10x reduction in inference costs. This highlights that algorithmic innovation, not just acquiring more hardware, will be a key competitive vector in the AI race, enabling more output with less energy.

The biggest performance breakthroughs in AI are not from isolated improvements in hardware, software, or models. They come from co-designing all three layers simultaneously, turning multiplicative 8x gains into exponential 100x gains, a concept Dylan Patel emphasizes as the key to leapfrogging innovation.

AI startup Wafer has demonstrated that with proper software optimization, AMD chips can achieve 80-100% of Nvidia's performance (in tokens per second) for specific open-source models, at roughly half the cost. This challenges the notion of Nvidia's insurmountable hardware dominance by proving software is a key performance unlock.

A key trend in AI models is "dynamism"—the ability to vary computation and memory usage per token, as seen in Mixture-of-Experts (MoE) architectures. Current hardware, designed before this trend, is inefficient. New chips must be built to accelerate these dynamic computations.

Building abstracted, cloud-agnostic AI stacks creates flexibility and eliminates vendor lock-in, but it introduces an impedance mismatch between software, runtime, networking, and silicon. Deep co-design across hardware and model architecture unlocks compounding 10% to 2x efficiency multipliers across every layer. Maximizing delivered intelligence per watt requires trading away some degree of fungibility in exchange for hardware-software co-optimization.

OpenAI achieved a major reduction in the cost of running its models through purely software and algorithmic improvements, such as quantization and smarter caching. This demonstrates that efficiency innovation can be as impactful as acquiring more hardware, suggesting a path to overcoming compute bottlenecks without relying solely on expensive chips.

Model architecture decisions directly impact inference performance. AI company Zyphra pre-selects target hardware and then chooses model parameters—such as a hidden dimension with many powers of two—to align with how GPUs split up workloads, maximizing efficiency from day one.

The impact of algorithmic progress is so immense that if we trained a model today using the same amount of compute as 2020's GPT-3, the resulting model would likely be more capable than 2023's much larger GPT-4. This highlights that compute isn't the only driver of progress.

Google's new state-of-the-art Deep Research agents are still powered by the older Gemini 3.1 Pro model. Their significant performance improvements come entirely from 'harness upgrades' and additional inference techniques. This demonstrates that the systems, tools, and processes surrounding a model are now a primary driver of capability, not just the raw power of the base model itself.