/
© 2026 RiffOn. All rights reserved.

Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

  1. Latent Space: The AI Engineer Podcast
  2. Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO
Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast · Oct 22, 2026

Modal CEO Erik Bernhardsson explains how serverless infrastructure transforms rigid multi-year GPU contracts into elastic AI compute.

Coding agents preserve deprecated syntax through stubborn training data

Erik Bernhardsson notes that teams struggle to deprecate APIs because coding agents continually generate outdated syntax absorbed during model training. As a result, infrastructure teams must make conservative changes and actively rewrite online docs, benchmarks, and public code samples to steer agent recommendations.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Centralized multi-tenant clouds outpace on-prem enterprise setups

Erik Bernhardsson considers on-premise deployments and private cloud VPC peering to be legacy security patterns grounded in outdated perimeter defenses. Controlling the entire cloud stack centrally allows infrastructure teams to deploy new databases, monitor hosts, and ship rapid updates without navigating delicate customer enterprise release cycles.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Prior generation cloud tools fail modern GPU workloads

Erik Bernhardsson points out that tools like Kubernetes and Docker were designed for predictable CPU backends with steady requests and low utilization costs. In contrast, modern generative AI requires moving large model weights across networks into expensive GPUs, demanding custom schedulers, snapshotting, and file systems.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Fast snapshotting turns agent sandboxes into ephemeral environments

Erik Bernhardsson observes that running agents inside permanent dev boxes creates wasteful compute overhead. By implementing rapid container snapshotting, developers can spin up a sandbox in one hundred milliseconds, execute a single command, snapshot state, and power down, allowing massive parallelization across agent tasks and test suites.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Forward deployed engineers must surface missing product features

Erik Bernhardsson expanded Modal's forward-deployed team from 4 to 24 engineers to help clients optimize models. Rather than operating as permanent consultants, these engineers focus on spotting product gaps. Their core mandate is translating recurring customer edge cases into permanent, automated platform features.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Decoupled inference providers smooth volatile AI compute demand

Erik Bernhardsson observes that when labs release superior models, user demand shifts rapidly away from older options. Individual model creators face erratic utilization, but independent inference providers easily adapt. They reallocate their shared pools of GPUs across different model providers as developer traffic swings.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Customer aggregation eliminates wasteful rectangular GPU commitments

Most cloud vendors sell GPU compute as rigid, three-year flat blocks starting months ahead, which Erik Bernhardsson notes does not fit real-world bursty AI workloads. By pooling thousands of customer workloads, an infrastructure provider smooths aggregate demand and serves dynamic compute on demand without requiring massive multi-year contracts.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Sudden hardware price drops pose bigger risks than GPU shortages

Erik Bernhardsson warns that buyers hoarding expensive multi-year GPU contracts face acute risks if hardware prices experience their historical downward pull. If prices drop, utilization dips and secondary resale markets disappear simultaneously, creating severe balance sheet shocks for startups that bought heavily at the top of the market.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Mandatory code reviews often signal broken automated testing

Erik Bernhardsson treats mandatory pull request reviews as an organizational crutch where developers dodge accountability for production deploys. Instead of stalling safe code changes, teams should make reviews optional and enforce system resilience through rigorous automated tests, decoupled architectures, and continuous deployment.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago

Neoclouds scale through debt backed by committed customer contracts

Erik Bernhardsson highlights how emerging cloud providers like Nscale rapidly build multi-billion-dollar footprints without owning initial hardware. They secure multi-year compute commitments from massive AI labs first, take those signed contracts to Wall Street lenders to finance hardware purchases, and deploy collateralized chips directly to buyers.

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO thumbnail

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Latent Space: The AI Engineer Podcast·a month ago