Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Specialized industrial machines struggle with items like napkins or laundry because deformable objects exhibit virtually infinite physical configurations. Adapting general vision-action models with brief task demonstrations reaches commercial quality faster than engineering single-purpose automation hardware.

Related Insights

Pre-training on massive human egocentric video gives foundation models a broad prior across diverse physical manipulation tasks. Fine-tuning with small batches of robot-specific data bridges embodiment gaps and establishes low-level dexterity without expensive massive teleoperation collection.

To build generalist robots, the most effective approach is pre-training foundation models on internet-scale video datasets, not just simulation or tele-operated data. This vast, diverse data provides a deep, implicit understanding of physics and object interaction that is impossible to replicate in controlled environments, enabling true generalization.

The Physical Intelligence thesis is that a foundation model learning from diverse data can achieve a "physical understanding" of the world, making it easier to adapt to new tasks than building single-purpose robots from scratch. Generality leverages broader data, which is ultimately a more scalable approach.

Building narrow AI creates high performance on one or two tasks, but the nth task costs as much to train as the first. In contrast, building generalist baselines endows models with broad common sense and physical reasoning. Once this broad base is established, tuning a robot to high reliability and mastery on any arbitrary task becomes significantly faster and cheaper.

Figure is observing that data from one robot performing a task (e.g., moving packages in a warehouse) improves the performance of other robots on completely different tasks (e.g., folding laundry at home). This powerful transfer learning, enabled by deep learning, is a key driver for scaling general-purpose capabilities.

Sunday Robotics found that as they scaled up pre-training data and compute for their laundry-folding robot, it developed the ability to learn a new task from a single demonstration. This suggests that complex abilities like one-shot learning don't need to be explicitly programmed but can emerge from scaled-up general training.

Robots can generalize skills to new hardware by using an intermediate 'thinking' step. The model generates an image of the next desired milestone (e.g., a half-folded shirt). This visual goal is embodiment-agnostic and easier to create than new motor commands, allowing the robot to then solve for the actions to match the image.

While "AI" is a common buzzword, the most significant recent advancement enabling flexible automation is the maturity of vision systems. These systems allow robots to identify and locate objects in a general space, removing the old constraint of needing perfectly pre-programmed, fixed coordinates for every action.

Manipulating deformable objects like towels was long considered one of the final, hardest challenges in robotics due to their infinite variations. The fact that Figure's neural networks can now successfully fold laundry indicates that the core technological hurdles for truly general-purpose robots have been overcome.

Beyond general training, world models enable a "real-to-sim-to-real" workflow. A user can capture a new environment with photos, instantly create a simulation, fine-tune a general-purpose robot for a specific task within that sim, and deploy it, enabling robot onboarding to new environments in minutes.

General Foundation Models Adapt to Deformable Object Manipulation Faster Than Dedicated Hardware | RiffOn