Unlike simple classification (one pass), generative AI performs recursive inference. Each new token (word, pixel) requires a full pass through the model, turning a single prompt into a series of demanding computations. This makes inference a major, ongoing driver of GPU demand, rivaling training.

Creating rich, interactive 3D worlds is currently so expensive it's reserved for AAA games with mass appeal. Generative spatial AI dramatically reduces this cost, paving the way for hyper-personalized 3D media for niche applications—like education or training—that were previously economically unviable.

The computational power for modern AI wasn't developed for AI research. Massive consumer demand for high-end gaming GPUs created the powerful, parallel processing hardware that researchers later realized was perfect for training neural networks, effectively subsidizing the AI boom.

The computational requirements for generative media scale dramatically across modalities. If a 200-token LLM prompt costs 1 unit of compute, a single image costs 100x that, and a 5-second video costs another 100x on top of that—a 10,000x total increase. 4K video adds another 10x multiplier.

NVIDIA's commitment to programmable GPUs over fixed-function ASICs (like a "transformer chip") is a strategic bet on rapid AI innovation. Since models are evolving so quickly (e.g., hybrid SSM-transformers), a flexible architecture is necessary to capture future algorithmic breakthroughs.

Instead of replacing entire systems with AI "world models," a superior approach is a hybrid model. Classical code should handle deterministic logic (like game physics), while AI provides a "differentiable" emergent layer for aesthetics and creativity (like real-time texturing). This leverages the unique strengths of both computational paradigms.

Traditional video models process an entire clip at once, causing delays. Descartes' Mirage model is autoregressive, predicting only the next frame based on the input stream and previously generated frames. This LLM-like approach is what enables its real-time, low-latency performance.

Instead of purely generative approaches, Moon Lake AI's strategy for creating interactive worlds involves using AI reasoning models to control and combine existing high-fidelity computer graphics tools. This is analogous to an LLM using a calculator, leveraging specialized tools for a more efficient and higher-quality outcome.

The primary performance bottleneck for LLMs is memory bandwidth (moving large weights), making them memory-bound. In contrast, diffusion-based video models are compute-bound, as they saturate the GPU's processing power by simultaneously denoising tens of thousands of tokens. This represents a fundamental difference in optimization strategy.