We don't perceive reality directly; our brain constructs a predictive model, filling in gaps and warping sensory input to help us act. Augmented reality isn't a tech fad but an intuitive evolution of this biological process, superimposing new data onto our brain's existing "controlled model" of the world.
Related Insights
Historically, computer vision treated 3D reconstruction (capturing reality) and generation (creating content) as separate fields. New techniques like NeRFs are merging them, creating a unified approach where models can seamlessly move between perceiving and imagining 3D spaces. This represents a major paradigm shift.
Our perception of sensing then reacting is an illusion. The brain constantly predicts the next moment based on past experiences, preparing actions before sensory information fully arrives. This predictive process is far more efficient than constantly reacting to the world from scratch, meaning we act first, then sense.
While LLMs dominate headlines, Dr. Fei-Fei Li argues that "spatial intelligence"—the ability to understand and interact with the 3D world—is the critical, underappreciated next step for AI. This capability is the linchpin for unlocking meaningful advances in robotics, design, and manufacturing.
Large language models are insufficient for tasks requiring real-world interaction and spatial understanding, like robotics or disaster response. World models provide this missing piece by generating interactive, reason-able 3D environments. They represent a foundational shift from language-based AI to a more holistic, spatially intelligent AI.
Instead of visually-obstructive headsets or glasses, the most practical and widely adopted form of AR will be audio-based. The evolution of Apple's AirPods, integrated seamlessly with an iPhone's camera and AI, will provide contextual information without the social and physical friction of wearing a device on your face.
World Labs argues that AI focused on language misses the fundamental "spatial intelligence" humans use to interact with the 3D world. This capability, which evolved over hundreds of millions of years, is crucial for true understanding and cannot be fully captured by 1D text, a lossy representation of physical reality.
A "frontier interface" is one where the interaction model is completely unknown. Historically, from light pens to cursors to multi-touch, the physical input mechanism has dictated the entire scope of what a computer can do. Brain-computer interfaces represent the next fundamental shift, moving beyond physical manipulation.
Current multimodal models shoehorn visual data into a 1D text-based sequence. True spatial intelligence is different. It requires a native 3D/4D representation to understand a world governed by physics, not just human-generated language. This is a foundational architectural shift, not an extension of LLMs.
AR and robotics are bottlenecked by software's inability to truly understand the 3D world. Spatial intelligence is positioned as the fundamental operating system that connects a device's digital "brain" to physical reality. This layer is crucial for enabling meaningful interaction and maturing the hardware platforms.
Human intelligence is multifaceted. While LLMs excel at linguistic intelligence, they lack spatial intelligence—the ability to understand, reason, and interact within a 3D world. This capability, crucial for tasks from robotics to scientific discovery, is the focus for the next wave of AI models.