Separating inference into "prefill" (memory-bound) and "decode" (bandwidth-bound) tasks is a game-changer for hardware longevity. It allows older GPUs to be used for prefill tasks indefinitely, extending their useful economic life from 3-4 years to 10-15 years, a boon for data centers and their financiers.

AI software is improving so rapidly that older hardware, like a three-year-old NVIDIA inference chip, is now more profitable than it was when new. This phenomenon, where software advancements outpace hardware depreciation, is unprecedented and makes existing infrastructure increasingly valuable.

Despite the rapid pace of hardware innovation, the value of older NVIDIA GPUs like the H100 is holding strong. Cloud provider CoreWeave reports these chips are retaining 90-95% of their pricing power over a 5-6 year lifespan because compute demand far outstrips supply.

Contrary to typical hardware depreciation, GPUs like NVIDIA's H100 are becoming more valuable over time. This is because newer, more efficient AI models can generate significantly more output and value on the same hardware, tying the GPU's worth to its utility rather than its age.

The massive investment in data centers isn't just a bet on today's models. As AI becomes more efficient, smaller yet powerful models will be deployed on older hardware. This extends the serviceable life and economic return of current infrastructure, ensuring today's data centers will still generate value years from now.

Andreessen highlights a unique economic phenomenon: the pace of AI software improvement outstrips hardware depreciation. This means a three-year-old NVIDIA inference chip can generate more revenue today than when it was new, a complete reversal of typical tech hardware value cycles.

Contrary to fears of rapid obsolescence, new domain-specific accelerators (DSAs) can be paired with older GPUs to handle specific tasks. This disaggregated approach extends the useful life of GPUs to 10-15 years, lowering financing costs for compute providers and invalidating bear cases.

Hyperscalers are extending depreciation schedules for AI hardware. While this may look like "cooking the books" to inflate earnings, it's justified by the reality that even 7-8 year old TPUs and GPUs are still running at 100% utilization for less complex AI tasks, making them valuable for longer and validating the accounting change.

Countering the narrative of rapid burnout, CoreWeave cites historical data showing a nearly 10-year service life for older NVIDIA GPUs (K80) in major clouds. Older chips remain valuable for less intensive tasks, creating a tiered system where new chips handle frontier models and older ones serve established workloads.