The MI300X's superior memory bandwidth and 192GB VRAM make it faster than H100s for non-FP8 dense transformers or MoE models. Quentin Anthony from Zyphra notes AMD's software has caught up, creating an under-appreciated arbitrage opportunity for teams willing to build on their stack.

While NVIDIA's CUDA software provides a powerful lock-in for AI training, its advantage is much weaker in the rapidly growing inference market. New platforms are demonstrating that developers can and will adopt alternative software stacks for deployment, challenging the notion of an insurmountable software moat.

The market undervalues chips from vendors like AMD because their software stack and kernel libraries are less mature than NVIDIA's CUDA. A team with deep expertise in low-level software and kernel optimization can extract significantly more performance from these chips, creating a powerful arbitrage opportunity by buying them at a discount.

OpenAI achieved a major reduction in the cost of running its models through purely software and algorithmic improvements, such as quantization and smarter caching. This demonstrates that efficiency innovation can be as impactful as acquiring more hardware, suggesting a path to overcoming compute bottlenecks without relying solely on expensive chips.

AMD competes with NVIDIA not just on GPU performance but by leveraging its wider range of CPUs. These are crucial for agentic AI workloads requiring many parallel processes, giving AMD an advantage over NVIDIA's more limited, GPU-focused CPU offerings.

To remain competitive, chip makers like AMD and Qualcomm must evolve beyond optimizing low-level kernels. The new battleground is a vertically integrated "intelligence layer"—offering their own highly-optimized foundation models tailored to their hardware. This strategy, pioneered by Nvidia with its NeMo framework, simplifies enterprise adoption.

NVIDIA's CUDA software, once its key advantage, is losing its grip. For inference, switching is trivial. More importantly, two of the three leading frontier models (from Google and Anthropic) were developed without CUDA, signaling a significant decline in its necessity for cutting-edge AI training.

Leading AI labs are moving beyond off-the-shelf hardware. They are now in a symbiotic co-design loop where an AI model's specific requirements inform the chip's architecture, and vice-versa. This tight integration of software and silicon is the new frontier for performance.

Previously, the bottleneck for AI labs was researcher time, making Nvidia's easy-to-use CUDA ecosystem dominant. Now, the biggest cost is compute capacity itself, creating massive economic incentives for labs to adopt cheaper, even if less convenient, competing chips from AMD or Google.