We scan new podcasts and send you the top 5 insights daily.
Interactive user chats provide multi-second human delays between queries, whereas deep horizon agentic workloads communicate autonomously in milliseconds. Because these agents constantly reason, fetch external state, and re-prompt models, demand shifts beyond raw accelerator compute into traditional infrastructure: orchestration CPUs, high-speed RAM, and storage. Data center topologies must adapt by managing the density and inter-building networking required between accelerator pods and CPU-heavy orchestration clusters.
The rise of agentic AI, which runs multiple parallel processes, is elevating the CPU's role from a secondary component to a critical 'conductor' for orchestrating GPU tasks, creating new demand and design considerations.
The rise of agentic AI and reinforcement learning is increasing the need for powerful CPUs located near GPUs. Cloud provider Nebius notes CPU requirements can be a high multiple of the GPU count, fueling a new demand cycle.
While GPUs are key for model training, the next AI wave of autonomous agents relies more on CPUs. The task of controlling and orchestrating multiple agents and tool calls is fundamentally a CPU-based process. This is creating a new hardware bottleneck and shifting focus to CPU manufacturers.
A new class of CPU is being designed for AI agents, which are always active, constantly feeding accelerators, and spawning thousands of sub-agents. These 'agentic CPUs' prioritize per-core memory and I/O bandwidth to coordinate the system, sacrificing legacy compatibility for maximum throughput and utilization.
The current AI boom focuses on GPUs for "thinking" (Gen AI). The next phase, "Agentic AI" for "doing," will rely heavily on CPUs for task orchestration and memory for context, creating new investment opportunities in this previously overshadowed hardware.
A 2-second delay is acceptable for a single user prompt. However, in an agentic system where 20 agents communicate sequentially, that delay compounds to 40 seconds, rendering the application unusable. This shift necessitates infrastructure with sub-second response times, driving hardware deployment to urban centers.
The focus on GPUs for AI overlooks a critical bottleneck: a growing CPU shortage. AI agents rely heavily on CPUs for orchestration tasks like tool calls, database queries, and web searches. This hidden demand is causing hyperscalers to lock in multi-year CPU supply contracts.
Beyond the well-known GPU scarcity for training AI, a massive CPU shortage looms for deployment. Providing a single AI agent for every knowledge worker would require 40 times the current annual global CPU production, highlighting a critical, under-discussed infrastructure challenge for the agentic web.
The transition from chatbots to autonomous 'agentic' AI represents a fundamental step-change. These agents, which execute complex tasks independently, have already increased the demand for computational power by 1000x, creating a massive, ongoing need for new infrastructure and hardware.
After the current memory crunch, the next AI infrastructure bottleneck will be CPU and networking. The complex orchestration required for emerging agentic AI systems will strain these resources, a trend already visible in companies like Fastly seeing demand spikes just for workload orchestration.