Human understanding is the ability to connect new information to a global, unified model of the universe. Until recently, AI models were isolated (e.g., a chess model). The major advance with large multimodal models is their ability to create a single, cohesive reality model, enabling true, generalizable understanding.

The next significant evolution in AI infrastructure is the shift to multimodal systems. Future tech stacks must move beyond single-modality paradigms (like text-only) to seamlessly handle and integrate text, images, audio, and video within a single, unified architecture.

Language is just one 'keyhole' into intelligence. True artificial general intelligence (AGI) requires 'world modeling'—a spatial intelligence that understands geometry, physics, and actions. This capability to represent and interact with the state of the world is the next critical phase of AI development beyond current language models.

The current focus on LLMs is a temporary phase. The true leap towards AGI will come from multi-sensory models that can process and integrate visual, auditory, and other data streams simultaneously, much like a human does. This moves AI from text generation to real-world understanding.

Large language models are insufficient for tasks requiring real-world interaction and spatial understanding, like robotics or disaster response. World models provide this missing piece by generating interactive, reason-able 3D environments. They represent a foundational shift from language-based AI to a more holistic, spatially intelligent AI.

Large Language Models are limited because they lack an understanding of the physical world. The next evolution is 'World Models'—AI trained on real-world sensory data to understand physics, space, and context. This is the foundational technology required to unlock physical AI like advanced robotics.

Manning counters LeCun's philosophy that language is just a "low bit rate" add-on. He posits that language, as a symbolic system, was the cognitive tool that vaulted human intelligence, enabling abstract reasoning and long-term planning—capabilities essential for advanced AI.

World Labs argues that AI focused on language misses the fundamental "spatial intelligence" humans use to interact with the 3D world. This capability, which evolved over hundreds of millions of years, is crucial for true understanding and cannot be fully captured by 1D text, a lossy representation of physical reality.

World Labs co-founder Fei-Fei Li posits that spatial intelligence—the ability to reason and interact in 3D space—is a distinct and complementary form of intelligence to language. This capability is essential for tasks like robotic manipulation and scientific discovery that cannot be reduced to linguistic descriptions.