Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Data centers don't contract for the maximum power listed on hardware ('plate rating'). Instead, they apply a derating factor to estimate realistic usage. Lightning AI uses a conservative 81% factor, which determines the actual contracted power and cost, preventing overpayment for unused capacity.

Related Insights

The standard for measuring large compute deals has shifted from number of GPUs to gigawatts of power. This provides a normalized, apples-to-apples comparison across different chip generations and manufacturers, acknowledging that energy is the primary bottleneck for building AI data centers.

The power consumption of AI data centers has ballooned from megawatts to gigawatts. Arista's CEO asserts that securing this level of power is a multi-year challenge, making it a larger and more immediate constraint on AI growth than the development of networking or compute technology itself.

The advertised per-hour GPU cost is misleading. Because research workloads are spiky and unpredictable, labs over-provision compute. This rampant underutilization means the effective price paid is often 10 times higher than the marketed rate, creating massive deadweight loss.

When power (watts) is the primary constraint for data centers, the total cost of compute becomes secondary. The crucial metric is performance-per-watt. This gives a massive pricing advantage to the most efficient chipmakers, as customers will pay anything for hardware that maximizes output from their limited power budget.

AI data centers are fundamentally different due to density. A single modern AI server consumes the power of an entire legacy rack (18kW). Additionally, fully-loaded cabinets can weigh over 4,200 pounds, making older raised-floor designs obsolete and requiring reinforced slab floors.

Data centers are ideal customers because they consume a steady, high amount of power, increasing the grid's overall utilization. Since electricity rates are total costs divided by kilowatt-hours delivered, adding these hyper-efficient customers increases the denominator, lowering the average rate for everyone.

Companies wanting to keep sensitive research data on-site are discovering a major infrastructure challenge. Even a small, local data center can double a lab facility's total power consumption, a critical and costly factor that must be planned for well in advance of securing space.

AI workloads, particularly for research and evals, don't follow predictable "follow-the-sun" patterns. They are extremely spiky, demanding massive compute resources instantly (e.g., 100,000 CPUs) and then dropping to zero. This forces providers like Daytona to maintain low mean utilization (15%) to handle unpredictable peaks.

The limiting factor for large-scale AI compute is no longer physical space but the availability of electrical power. As a result, the industry now sizes and discusses data center capacity and deals in terms of megawatts, reflecting the primary constraint on growth.

The energy demands of modern AI are difficult to contextualize. A one-gigawatt data center uses as much power as a city of nearly one million US households. A five-gigawatt facility requires a 5,000-acre building footprint, excluding any power infrastructure.

AI Data Centers Use an 81% "Derating" Factor on Equipment Power Ratings for Realistic Costing | RiffOn