Startups and major labs are focusing on "world models," which simulate physical reality, cause, and effect. This is seen as the necessary step beyond text-based LLMs to create agents that can truly understand and interact with the physical world, a key step towards AGI.

A core challenge in physical AI is the tension between large, powerful models (offboard, in a data center) and the need for low-latency models (onboard, on the machine). The key is using techniques like distillation to create smaller derivatives that run in milliseconds for safety-critical decisions.

Waymo’s system starts with a large, off-board foundation model understanding the physical world. This is specialized into three 'teacher' models: the Driver, the Simulator, and the Critic. These teachers then train smaller, efficient 'student' models that run in the vehicle.

Beyond generative AI, Lip-Bu Tan sees a massive opportunity in 'physical AI' for robotics and autonomous systems. Winning here requires more than just powerful chips; it demands a full-stack solution with co-designed hardware (XPU), software, and advanced packaging, all tailored for specific physical workloads.

While often used interchangeably, 'Physical AI' is more specific than 'Edge AI.' Edge AI broadly concerns processing data locally. Physical AI refers to edge systems, like robots or autonomous vehicles, that not only sense and predict but also execute physical actions based on those predictions.

Waymo uses a foundation model to create specialized, high-capacity "teacher" models (Driver, Simulator, Critic) offline. These teachers then distill their knowledge into smaller, efficient "student" models that can run in real-time on the vehicle, balancing massive computational power with on-device constraints.

NVIDIA conceptualizes the AV challenge around three distinct computing pillars: a training computer for models, a simulation computer for validation, and an in-car inference computer for real-time decisions. This framework highlights the massive, multi-faceted compute investment required for full autonomy.

To ensure safety, NVIDIA runs two software stacks in its cars. One is the end-to-end AI model making driving decisions. The other is a "classical stack," a traditional, component-based system that acts as a real-time safety guardrail, constantly verifying the AI's trajectory outputs frame-by-frame.

Unlike text agents that tolerate multi-second latency, embodied agents handling real-world objects cannot wait on cloud inference without risking physical failure or safety hazards. Deployments require training large models and distilling them down for local execution.