Despite massive compute demand from frontier AI labs willing to buy out all supply, AWS intentionally reserves GPU and accelerator capacity for early-stage startups. Matt Garman estimates that 30% to 40% of AWS's current revenue comes from companies that began as startups on AWS. Sacrificing short-term lab revenue ensures a healthy ecosystem and guarantees access to the next generation of enterprise customers.
Agentic workflows challenge traditional enterprise cloud assumptions. Matt Garman notes that autonomous agents are uniquely sensitive to P999 latency, unlike human developers who tolerate slight delays. Furthermore, agents frequently spin up databases for transient tasks, making standard multi-region 'five nines' durability over-engineered and resource-heavy. Cloud providers must rethink primitives to support high-speed, disposable infrastructure alongside durable storage.
While some neoclouds have 30% to 60% customer revenue concentration with a few frontier labs, AWS caps individual customer allocations at single-digit percentages. Matt Garman explains that AWS actively rations compute across frontier labs, enterprises, and startups. This diversification shields AWS from potential AI industry downturns while ensuring their capacity is backed by real, production-level workloads that demonstrate immediate positive return on investment.
Enterprises struggle with agent evaluation, data labeling, and drift detection, creating massive demand for Forward Deployed Engineers (FDEs). Matt Garman argues that rather than creating indefinite consulting dependencies, FDE engagements should be strictly capped around 45 days. The objective is to co-develop evaluation frameworks and upskill enterprise teams to operate and own their agent pipelines independently.
Inside AWS, engineering teams are transitioning from writing code to managing fleets of agents. Matt Garman notes this has shrunk team sizes needed per initiative from ten engineers down to three or four. Because these smaller teams deliver features significantly faster, organizations face the operational challenge of continually redeploying engineers across projects while sustaining ongoing maintenance.
Data center scaling does not face a single structural constraint; rather, bottlenecks rotate continuously across tiers. Matt Garman points out that constraints regularly shift from chip fabrication to power generation, grid transmission, memory supplies, and even local construction labor shortages. Consequently, hyperscalers must plan multi-year component pipelines four to five tiers deep in the supply chain and directly procure renewable and nuclear power.
Hardware architectures originally built for AI training can outcompete purpose-built inference hardware as model sizes scale. AWS originally built Inferentia specifically for inference and Trainium for training. However, Garman notes that as enterprise models grew significantly, Trainium actually emerged as their highest-performing and most cost-effective inference chip, powering the vast majority of Bedrock's operational inference workloads today.
Traditional Identity and Access Management (IAM) permissions and service roles are ill-suited for autonomous agents. According to Matt Garman, granting an agent full human credentials introduces severe risks of accidental deletion or operational disruption. Instead, agent infrastructure requires specialized building blocks: microVM sandboxes (like Firecracker), time-boxed temporary permissions, and fine-grained tool restrictions that constrain agent execution to specific, isolated tasks.