The AI supply chain is crunched not just by obvious components like TSMC wafers and HBM memory. A significant, often overlooked bottleneck is rack manufacturing—including high-speed cables, connectors, and even sheet metal—which are "sneaky hard" due to extreme power, heat, and signal integrity demands.

The short range of copper cables is a key driver behind modern data center design. To maintain bandwidth, GPUs are packed into incredibly dense, megawatt racks. These racks are so heavy they require reinforced concrete floors to support their weight, highlighting a physical bottleneck that photonics technology aims to solve.

Increasing the number of GPUs in a high-speed "scale-up" domain is a physical engineering challenge. It's constrained by the sheer density of cables that can fit within a rack's backplane, along with factors like cable bend radius, power delivery, cooling capacity, and structural weight.

While AI inference can be decentralized, training the most powerful models demands extreme centralization of compute. The necessity for high-bandwidth, low-latency communication between GPUs means the best models are trained by concentrating hardware in the smallest possible physical space, a direct contradiction to decentralized ideals.

Andrew Feldman, CEO of competitor Cerebras, argues their single wafer-scale chip is superior for large AI models. He contends that connecting thousands of smaller GPUs, as Nvidia does, introduces significant latency from physical wiring that negates on-paper performance specs, creating a fundamental bottleneck.

The quest for nanosecond advantages is a physical battle over geography. It began with co-locating servers in data centers, escalated to digging dedicated, straighter fiber optic cables from Chicago to New Jersey, and culminated in building microwave tower networks for even faster, line-of-sight data transmission.

With Moore's Law over, computing progress now depends on networking vast numbers of chips. Lightmatter's photonic interconnects overcome the distance limits of copper cables, allowing thousands of GPUs kilometers apart to function as a single, cohesive supercomputer. This creates a new scaling vector for AI performance.

AI networking is not an evolution of cloud networking but a new paradigm. It's a 'back-end' system designed to connect thousands of GPUs, handling traffic with far greater intensity, durability, and burstiness than the 'front-end' networks serving general-purpose cloud workloads, requiring different metrics and parameters.

Mixture-of-Experts (MoE) models require an "all-to-all" communication pattern. This is efficient within a single GPU rack's high-speed interconnect but becomes a major bottleneck between racks, where communication is ~8x slower. This effectively limits an MoE layer's maximum size to what a single rack can support.