As GPU data transfer speeds escalate, traditional electricity-based communication between nearby chips faces physical limitations. The industry is shifting to optics (light) for this "scale-up" networking. Nvidia is likely to acquire a company like IR Labs to secure this photonic interconnect technology, crucial for future chip architectures.

The core architectural bet for Cerebras was that incremental improvements on an existing design (like a GPU) yield minimal gains because the incumbent has already optimized it. To achieve a step-change in performance, a fundamentally different approach is required, leading them to their massive, wafer-scale chip design.

As AI hardware hits physical limits, photonics (using light for data transfer) is emerging as the critical solution. It simultaneously addresses three major constraints: power consumption, cooling requirements, and data transfer speeds. This makes it a uniquely powerful technology that will enable the next generation of AI infrastructure.

With Moore's Law dead, Shkreli argues the future of computing lies in photonics. Using light for matrix multiplication (MATMOLs) offers a theoretical 1,000x to 1,000,000x performance gain, making it the necessary next frontier despite major technical hurdles.

The plateauing performance-per-watt of GPUs suggests that simply scaling current matrix multiplication-heavy architectures is unsustainable. This hardware limitation may necessitate research into new computational primitives and neural network designs built for large-scale distributed systems, not single devices.

Martin Shkreli makes a case for photonic computing—using light instead of electrons—as the next major paradigm in AI hardware. He argues that because matrix multiplications (95% of a GPU's job) are a natural function of light interference, photonic chips could offer an "insane speedup" with O-of-one complexity, making them a potential successor to GPUs.

With Moore's Law over, computing progress now depends on networking vast numbers of chips. Lightmatter's photonic interconnects overcome the distance limits of copper cables, allowing thousands of GPUs kilometers apart to function as a single, cohesive supercomputer. This creates a new scaling vector for AI performance.

GPUs were designed for graphics, not AI. It was a "twist of fate" that their massively parallel architecture suited AI workloads. Chips designed from scratch for AI would be much more efficient, opening the door for new startups to build better, more specialized hardware and challenge incumbents.

Today's transformers are optimized for matrix multiplication (MatMul) on GPUs. However, as compute scales to distributed clusters, MatMul may not be the most efficient primitive. Future AI architectures could be drastically different, built on new primitives better suited for large-scale, distributed hardware.