The primary challenge in robotics AI is the lack of real-world training data. To solve this, models are bootstrapped using a combination of learning from human lifestyle videos and extensive simulation environments. This creates a foundational model capable of initial deployment, which then generates a real-world data flywheel.

Figure trains its robot's stability controller entirely in a physics simulator, akin to a video game. This allows them to test countless scenarios synthetically. The resulting AI model is so effective it can be 'zero-shot' deployed directly onto the physical robot, achieving human-level stability immediately.

While language models understand the world through text, Demis Hassabis argues they lack an intuitive grasp of physics and spatial dynamics. He sees 'world models'—simulations that understand cause and effect in the physical world—as the critical technology needed to advance AI from digital tasks to effective robotics.

Siemens discovered that standard virtual training for robots was insufficient for real-world application. The robot's accuracy only jumped to a usable level after they switched to a photorealistic digital twin using advanced ray-tracing, which more accurately modeled light and texture for the AI.

Large language models are insufficient for tasks requiring real-world interaction and spatial understanding, like robotics or disaster response. World models provide this missing piece by generating interactive, reason-able 3D environments. They represent a foundational shift from language-based AI to a more holistic, spatially intelligent AI.

The AI's ability to handle novel situations isn't just an emergent property of scale. Waive actively trains "world models," which are internal generative simulators. This enables the AI to reason about what might happen next, leading to sophisticated behaviors like nudging into intersections or slowing in fog.

Generalist CEO Pete Florence provides a tier list for robotics training data. He ranks "lived experience of the physical world" as S-tier, emphasizing the irreplaceable value of high-quality, real-world data. In contrast, he rates synthetic data from world models as F-tier, suggesting it is far less effective.

Robots have become so capable at low-level physical tasks that the primary bottleneck has shifted to "mid-level reasoning"—interpreting a scene and choosing the correct next action. This means improvement can come from high-level language-based coaching, not just more physical demonstration data, which is a major breakthrough.

Instead of using traditional, rule-based simulators, Comma AI trains its driving agent inside a learned "world model." This generative model creates photorealistic, diverse driving scenarios and, crucially, responds accurately to the agent's simulated actions—a key requirement for effective robotics training.