Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Jensen Huang explains that step-by-step reasoning, planning, and maintaining strategic optionality demand far more inferences than simply regurgitating memorized training data. As models become more capable at solving complex problems, organizations assign them more tasks, compounding aggregate compute consumption.

Related Insights

Contrary to the view that AI token intensity will drop after the initial coding boom, the move from simple queries to autonomous 'agentic' workflows will cause an order-of-magnitude (10x) increase in token usage per task. This applies across all knowledge-based jobs, ensuring sustained and explosive demand for compute.

The shift from simple chatbots (one user request, one API call) to agentic AI systems will decouple inference requests from direct user actions. A single user request could trigger hundreds or thousands of automated model calls, leading to an exponential increase in compute demand and cost.

The next leap in AI's value will come from "agents" that work on problems autonomously without direct, real-time user commands. This shift from a reactive, search-engine-like model to a proactive, problem-solving one will drive a 5x increase in compute consumption and unlock new applications.

While the cost to achieve a fixed capability level (e.g., GPT-4 at launch) has dropped over 100x, overall enterprise spending is increasing. This paradox is explained by powerful multipliers: demand for frontier models, longer reasoning chains, and multi-step agentic workflows that consume exponentially more tokens.

The shift from simple query-based AI to agentic AI, where AI calls itself recursively to solve complex tasks, increases compute demand by orders of magnitude. Most people, especially non-coders, fail to grasp this exponential shift, leading them to consistently underestimate the scale and duration of the AI infrastructure build-out.

AI's computational needs are not just from initial training. They compound exponentially due to post-training (reinforcement learning) and inference (multi-step reasoning), creating a much larger demand profile than previously understood and driving a billion-X increase in compute.

The next wave of AI adoption involves 'agentic' workflows, where AI performs complex tasks autonomously. This shift from simple queries to agentic use is expected to increase token consumption by approximately 10x per task. This will drive a massive explosion in compute demand across all knowledge-work industries, not just coding.

Jensen Huang quantifies the massive computational leap required for advanced AI. The move from generative AI to reasoning was a 100x compute increase, and the subsequent move to agentic systems that can perform work represents another 100x jump. This results in a staggering 10,000x increase in computational demand in just two years.

The transition from chatbots to autonomous 'agentic' AI represents a fundamental step-change. These agents, which execute complex tasks independently, have already increased the demand for computational power by 1000x, creating a massive, ongoing need for new infrastructure and hardware.

As AI moves from simple chatbots to complex agents, its demand for computation grows exponentially. This is because advanced techniques like chain-of-thought reasoning and using AI to generate more efficient code fundamentally involve using massive amounts of inference to achieve better results, creating a feedback loop.