Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Agentic workflows challenge traditional enterprise cloud assumptions. Matt Garman notes that autonomous agents are uniquely sensitive to P999 latency, unlike human developers who tolerate slight delays. Furthermore, agents frequently spin up databases for transient tasks, making standard multi-region 'five nines' durability over-engineered and resource-heavy. Cloud providers must rethink primitives to support high-speed, disposable infrastructure alongside durable storage.

Related Insights

The internet's next chapter moves beyond serving pages to executing complex, long-duration AI agent workflows. This paradigm shift, as articulated by Vercel's CEO, necessitates a new "AI Cloud" built to handle persistent, stateful processes that "think" for extended periods.

Interactive user chats provide multi-second human delays between queries, whereas deep horizon agentic workloads communicate autonomously in milliseconds. Because these agents constantly reason, fetch external state, and re-prompt models, demand shifts beyond raw accelerator compute into traditional infrastructure: orchestration CPUs, high-speed RAM, and storage. Data center topologies must adapt by managing the density and inter-building networking required between accelerator pods and CPU-heavy orchestration clusters.

The core needs of AI agents—version control, testing, observability—mirror those of human developers. However, the sheer scale and speed of agentic workflows mean existing tools like Kubernetes are insufficient, requiring a fundamental reimagining of the entire infrastructure stack.

To run reliably in the cloud, AI agents cannot be simple synchronous API calls. Their long-running, stateful nature requires an asynchronous architecture. This typically involves a message broker and task queue to farm out agentic loops to ephemeral workers, preventing process failures and enabling scalability.

The dominant AI use case will shift from real-time, human-in-the-loop chatbots to long-running background agents. For these agents, which work for hours or days, an extra few seconds of latency is meaningless, unlocking massive cost-saving opportunities by prioritizing throughput over speed.

Running multiple, complex AI coding agents simultaneously is computationally prohibitive on local machines. Stripe's success relies on their ability to spin up numerous isolated cloud development environments in parallel, a crucial investment for any team serious about agentic engineering.

A 2-second delay is acceptable for a single user prompt. However, in an agentic system where 20 agents communicate sequentially, that delay compounds to 40 seconds, rendering the application unusable. This shift necessitates infrastructure with sub-second response times, driving hardware deployment to urban centers.

The need for high-availability data centers is an assumption from the training and real-time era. For asynchronous background agents, a distributed fleet of small, cheap data centers with 95% uptime is viable. Failures are handled by a robust control plane that reroutes work, trading P99 latency for unbeatable economics.

The rise of AI agents drives demand for a new computing primitive: secure, small-footprint virtual machines that can start in milliseconds, execute a task, persist state, and then sleep. This optimizes CPU usage for the high-volume, short-burst workflows characteristic of agents.

While local coding agents have product-market fit today, OpenAI's Michael Bolin argues the long-term trend is remote agents. To achieve true automation—like having an agent autonomously tackle every new bug ticket—workloads must run in the cloud, unconstrained by a developer's personal machine.