We scan new podcasts and send you the top 5 insights daily.
Josh Gao says scaling the current data recipe by orders of magnitude will solve many constrained tasks. But a drop-in robot that handles any task amid real-world complexity, like weather, is still far off; he expects tactile, simulation, and even smell-like modalities to enter omni-model training.
To build generalist robots, the most effective approach is pre-training foundation models on internet-scale video datasets, not just simulation or tele-operated data. This vast, diverse data provides a deep, implicit understanding of physics and object interaction that is impossible to replicate in controlled environments, enabling true generalization.
Josh Gao says it is expected that models like GPT-6 Astra generalize to robot tasks because robotics data and action trajectories are being added to pre-training mixes of large foundation models. He still sees room for specialized robot models that reach four-nines reliability in narrow tasks.
Language models execute identically regardless of device, but robotics software is constrained by the physical robot body. Keerthana Gopalakrishnan places current robotics in the 'GPT-2 era' because generic robotic brains cannot reliably control arbitrary, unseen robot embodiments out of the box, nor can they reliably generalize from few-shot demonstrations across hard, diverse tasks.
Simply collecting more data from a deployed robot isn't enough to create a powerful learning flywheel. If the tasks are repetitive (e.g., a million car welds), the model won't generalize. Data must be diverse, acting more like an 'education program' than a fungible commodity to drive real capability growth.
Neurobotics posits that true physical AI requires more than just vision-language models; it needs a "nervous system" and reflexes. They advocate for training robots in physical "gyms" to collect embodied data, arguing that complex physical tasks cannot be learned solely by watching videos.
No single data collection method will solve robotics. Teleoperation provides high-precision ground truth for specific robots but scales poorly and degrades when hardware updates. Universal Manipulation Interface (UMI) data offers better scale and precision via sensors but requires hardware maintenance. Egocentric human video provides massive scale but lacks precise end-effector force signals and suffers from human-to-robot physical embodiment mismatch.
Unlike LLMs trained on vast digital text, humanoid robots need immense amounts of real-world physical data to learn simple tasks. It's estimated that 100 million hours—over 11,000 years' worth—of interaction data is needed to create truly smart, useful humanoids, highlighting the scale of the challenge.
Despite industry hype, humanoid robots are not imminent. They lack the massive datasets of real-world, unpredictable interactions needed to operate safely and usefully in a home environment, which is far more complex than a structured factory floor.
The "bitter lesson" (scale and simple models win) works for language because training data (text) aligns with the output (text). Robotics faces a critical misalignment: it's trained on passive web videos but needs to output physical actions in a 3D world. This data gap is a fundamental hurdle that pure scaling cannot solve.
For physical AI, the primary constraint is not the cost of data but its fundamental non-existence. Unlike software AI, you can't advance without deploying robots "in the wild" to capture edge cases—a classic chicken-and-egg problem that simulation alone cannot solve and capital cannot easily buy.