We scan new podcasts and send you the top 5 insights daily.
Sergey Levine explains that solving all robotics tasks at once works better than training specialized models, mirroring how general language models replaced single-purpose translation systems. Generality exposes the model to edge cases and varied data, turning what seems like an overwhelming scope into a learning advantage.
To build generalist robots, the most effective approach is pre-training foundation models on internet-scale video datasets, not just simulation or tele-operated data. This vast, diverse data provides a deep, implicit understanding of physics and object interaction that is impossible to replicate in controlled environments, enabling true generalization.
Josh Gao says it is expected that models like GPT-6 Astra generalize to robot tasks because robotics data and action trajectories are being added to pre-training mixes of large foundation models. He still sees room for specialized robot models that reach four-nines reliability in narrow tasks.
The Physical Intelligence thesis is that a foundation model learning from diverse data can achieve a "physical understanding" of the world, making it easier to adapt to new tasks than building single-purpose robots from scratch. Generality leverages broader data, which is ultimately a more scalable approach.
Building narrow AI creates high performance on one or two tasks, but the nth task costs as much to train as the first. In contrast, building generalist baselines endows models with broad common sense and physical reasoning. Once this broad base is established, tuning a robot to high reliability and mastery on any arbitrary task becomes significantly faster and cheaper.
Figure is observing that data from one robot performing a task (e.g., moving packages in a warehouse) improves the performance of other robots on completely different tasks (e.g., folding laundry at home). This powerful transfer learning, enabled by deep learning, is a key driver for scaling general-purpose capabilities.
Sergey Levine points out that autonomous cars required seventeen years to reach scale because driving is an unforgiving, high-stakes task with no margin for playful experimentation. In contrast, general-purpose robots can acquire core physical skills on low-stakes household tasks before transferring those capabilities to dangerous environments.
Adopting the true 'foundation model ethos' from LLMs is difficult for roboticists. It means a warehouse automation company should collect data from kitchen robots. This breadth, while seemingly unrelated, builds a generalist model that better handles the weird edge cases in the target domain than a narrowly trained specialist model.
Sunday Robotics found that as they scaled up pre-training data and compute for their laundry-folding robot, it developed the ability to learn a new task from a single demonstration. This suggests that complex abilities like one-shot learning don't need to be explicitly programmed but can emerge from scaled-up general training.
For unpredictable situations where a robot has no prior training data (e.g., a "gas leak" sign), multimodal LLMs can provide the necessary world knowledge to reason and act appropriately. This solves the long-standing robotics problem of how to handle the long tail of real-world scenarios.
Unlike older robots requiring precise maps and trajectory calculations, new robots use internet-scale common sense and learn motion by mimicking humans or simulations. This combination has “wiped the slate clean” for what is possible in the field.