The long-standing trend of centralizing all data into a single warehouse is incompatible with the speed of AI. Large-scale data migrations are too slow. The future architecture will involve AI models operating closer to data sources for faster, decentralized operation.

The need for low-latency services for agents and real-time applications in finance and healthcare is driving a shift towards distributed data centers. Instead of remote gigawatt facilities, companies are deploying smaller, power-efficient, air-cooled racks like SambaNova's in existing metropolitan data centers, closer to users.

The U.S. has plenty of power for the AI boom, but it's in the wrong places—far from existing data centers, fiber networks, and population centers. The critical challenge is not generation capacity but rather bridging the geographical gap between where power is abundant and where it is needed.

While AI training requires massive, centralized data centers, the growth of inference workloads is creating a need for a new architecture. This involves smaller (e.g., 5 megawatt), decentralized clusters located closer to users to reduce latency. This shift impacts everything from data center design to the software required to manage these distributed fleets.

The initial assumption of a centralized AI model (large hub, large spoke) is wrong. The new model will involve large foundational hubs, enterprise-specific training hubs, and distributed "spokes" of on-premise hardware for inference. This shift is driven by the need for data control and cost efficiency.

The need for high-availability data centers is an assumption from the training and real-time era. For asynchronous background agents, a distributed fleet of small, cheap data centers with 95% uptime is viable. Failures are handled by a robust control plane that reroutes work, trading P99 latency for unbeatable economics.

The primary factor for siting new AI hubs has shifted from network routes and cheap land to the availability of stable, large-scale electricity. This creates "strategic electricity advantages" where regions with reliable grids and generation capacity are becoming the new epicenters for AI infrastructure, regardless of their prior tech hub status.

Unlike physical commodities like oil, AI compute lacks strong regional price differences. For non-real-time tasks like model training, the physical location of the GPU and its associated latency are negligible. This allows users to source compute globally, driving prices toward a single international benchmark.

Microsoft's new data centers, like Fairwater 2, are designed for massive scale. They use high-speed networking to aggregate computing power across different sites and even regions (e.g., Atlanta and Wisconsin), enabling training of unprecedentedly large models on a single job.