Waymo demonstrated that a standard Vision Language Model (VLM) can be fine-tuned to output driving trajectories instead of text. While unsafe for public roads, it drives 'pretty darn well' in normal conditions, showing the surprising generalizability of foundational vision-language understanding.

Unlike video generation models that merely predict pixels, Moonlake argues a true world model must understand and predict the consequences of actions over time. This requires an abstracted, semantic understanding of the world, not just visual fidelity.

To build generalist robots, the most effective approach is pre-training foundation models on internet-scale video datasets, not just simulation or tele-operated data. This vast, diverse data provides a deep, implicit understanding of physics and object interaction that is impossible to replicate in controlled environments, enabling true generalization.

Language is a human-optimized construct, but the visual world is not. It contains a "fat tail" of chaotic scenes that are harder for models to learn, explaining why vision capabilities today resemble natural language processing from the GPT-3 era.

A key trend to watch is the rise of Vision-Language-Action (VLA) models, which are critical for robotics. These models take an instruction (language), understand a scene (vision), and then manipulate the environment (action). This represents a new paradigm that combines "read" and "write" access to the physical world, often requiring edge-ready compute.

The next frontier for visual intelligence is twofold: creating truly multimodal models that retain long-term context of user interactions without re-prompting, and developing real-time generation. Real-time capabilities are crucial for creating duplex interactions and enabling robots to perceive and act instantly.

The concept of a 'world model' is evolving from action-conditioned video predictors to single, multimodal models like Google's Omni. Omni demonstrates a deep, scalable understanding of the world, shown through nuanced video editing, representing a more practical approach than traditional, computationally expensive architectures.

World Labs argues that AI focused on language misses the fundamental "spatial intelligence" humans use to interact with the 3D world. This capability, which evolved over hundreds of millions of years, is crucial for true understanding and cannot be fully captured by 1D text, a lossy representation of physical reality.

Robotic World Action Models output predicted future visual states before executing physical movements. Generating representations of future world states serves as visual reasoning in physics space, guiding precise actions and improving generalization in unfamiliar environments.