The ultra-fast 'East-West' networks connecting GPUs have strict physical limits. Signals must travel between any two GPUs in a cluster within picoseconds, limiting cable lengths to a few hundred meters. This physical constraint directly dictates the layout and maximum size of a single contiguous data hall.

As NVIDIA moves to massive rack-scale systems, the primary installation challenge has evolved. It's no longer just about the chips, but the immense cabling and networking connecting them. Diagnosing a single failed cable among kilometers of wiring is now the crucial, non-linear problem, described by one expert as "black magic."

Theoretical FLOPS measure maximum compute potential in isolation, ignoring real-world operating friction. At scales of 100,000 accelerators, bleeding-edge components fail multiple times per day or hour. Because synchronous workloads halt entirely when a single node drops, progress depends on checkpointing, debugging, and recovery speed. Google evaluates performance using 'goodput'—the net useful work delivered toward the actual workload under real failure conditions rather than raw theoretical throughput.

While fiber optics long dominated data transfer between data centers ('scale out'), the next growth wave is 'scale up.' This involves replacing short copper Ethernet cables within individual server racks, a move necessitated by rising data speeds and the physical limitations of copper, promising unprecedented unit volumes.

As AI hardware hits physical limits, photonics (using light for data transfer) is emerging as the critical solution. It simultaneously addresses three major constraints: power consumption, cooling requirements, and data transfer speeds. This makes it a uniquely powerful technology that will enable the next generation of AI infrastructure.

With Moore's Law over, computing progress now depends on networking vast numbers of chips. Lightmatter's photonic interconnects overcome the distance limits of copper cables, allowing thousands of GPUs kilometers apart to function as a single, cohesive supercomputer. This creates a new scaling vector for AI performance.

The need for high-availability data centers is an assumption from the training and real-time era. For asynchronous background agents, a distributed fleet of small, cheap data centers with 95% uptime is viable. Failures are handled by a robust control plane that reroutes work, trading P99 latency for unbeatable economics.

When splitting jobs across thousands of GPUs, inconsistent communication times (jitter) create bottlenecks, forcing the use of fewer GPUs. A network with predictable, uniform latency enables far greater parallelization and overall cluster efficiency, making it more important than raw 'hero number' bandwidth.

When building systems with hundreds of thousands of GPUs and millions of components, it's a statistical certainty that something is always broken. Therefore, hardware and software must be architected from the ground up to handle constant, inevitable failures while maintaining performance and service availability.