Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

Etched's young founders faced universal disbelief. They succeeded by challenging industry dogma—like default chip temperature settings—and persistently demonstrating their technical validity to respected veterans, eventually converting them into key hires like their CTO.

Adding more FLOPS to current AI chips is useless due to thermal throttling. Etched realized the solution is lowering voltage, which quadratically reduces power consumption. Inspired by bitcoin miners, they created a new power delivery system enabling chips to run at under half the voltage of GPUs.

Instead of focusing on on-chip memory bandwidth, Etched optimized for cluster-scale memory. They built a custom interconnect that cuts chip-to-chip latency by over 5x compared to GPUs. This allows the memory of the entire cluster to function as a single, low-latency pool, dramatically improving performance.

Etched builds its own chips, boards, cold plates, interconnects, and even its own racks. This full-stack ownership allows for extreme parallelization and iteration speed, a key advantage over startups that rely on a fragmented supply chain and multiple vendors.

Etched's hiring philosophy is bimodal. They recruit "legends"—the world's best in a specific domain—and pair them with brilliant, inexperienced young talent. This combination provides both deep expertise and the aggressive, first-principles questioning needed to challenge industry norms.

Etched's seemingly impossible mission—two 24-year-olds taking on NVIDIA—acts as a natural recruiting filter. It deters opportunistic candidates and attracts those who are wired to take on extreme challenges, self-selecting for a team that is personally invested in proving the vision right.

Facing a year-long delay from a key vendor, Etched relocated a dozen of its top engineers to the vendor's office in Bangalore for six months. This extreme, hands-on intervention allowed them to run 24-hour development cycles and ship on time while competitors stalled.

Etched uses a strategy called "prefetching" to compress timelines. Before their silicon arrived, they built racks with mock thermal chips and ran their full software stack on FPGAs. This ensured everything was ready the moment the real chips landed, collapsing their bring-up time.

People focus on TSMC's leading-edge tech, but its key differentiator is customer service. The company actively partners with clients, running experiments at its own cost to help them improve yields. This collaborative approach is a powerful, often overlooked, competitive advantage.

Instead of building a generic graph compiler, Etched focused on hand-optimized kernels. This approach, similar to high-frequency trading firms, provides maximum performance. It's also future-proof, as they design their tools for AI models to use directly, anticipating a time when AI writes its own kernels.

AI chip projects at Google, Meta, or OpenAI are not existential; the companies will survive if they fail. This creates a risk-averse culture. A dedicated startup like Etched, whose entire existence depends on its chip's success, is incentivized to take bigger risks to create a superior product.

A key trend in AI models is "dynamism"—the ability to vary computation and memory usage per token, as seen in Mixture-of-Experts (MoE) architectures. Current hardware, designed before this trend, is inefficient. New chips must be built to accelerate these dynamic computations.