It's surprising that AI models trained on general data can accurately predict rare events like hurricanes. The reason is that the physical world is "forgiving"; extreme phenomena are governed by strong physical structures and signatures that AI can learn effectively, even from a limited number of examples.

Unlike traditional neural networks which require fixed-resolution inputs (e.g., pixels), neural operators model data as continuous functions. This allows them to "zoom in" and make predictions at resolutions higher than the training data, a crucial capability for multi-scale physical phenomena like weather patterns.

Goodfire's research on 'neural geometry' reveals that concepts inside models have distinct, low-dimensional shapes (manifolds). For example, numbers form a helix and temperature forms a spiral arc. Understanding these shapes allows for more precise and effective interventions, moving beyond linear vector manipulations.

Counter-intuitively, successful weather and climate AI models are not trained on long-term data. They are trained to predict only the next six hours autoregressively. This surprisingly generalizes to stable rollouts predicting weather patterns for hundreds or thousands of steps into the future, enabling long-term forecasting from short-term training.

Traditional weather forecasting requires massive supercomputers, limiting access to large agencies. New AI models are tens of thousands of times faster and can run on a single consumer-grade GPU. This democratizes high-fidelity weather modeling for smaller agencies and nations, especially in the global south.

Large Language Models are limited because they lack an understanding of the physical world. The next evolution is 'World Models'—AI trained on real-world sensory data to understand physics, space, and context. This is the foundational technology required to unlock physical AI like advanced robotics.

PINs, which solve PDEs from scratch using only physics constraints, often fail on complex, time-dependent problems due to difficult optimization landscapes. Neural operators overcome this by using a data-driven, supervised learning approach, learning from existing solutions before applying physics constraints, making them more robust.

While early AI development requires constant testing of new models, Conative.ai found they eventually reached a stable architecture. The focus then shifted from wholesale model replacement to fine-tuning existing layers with specific data, reducing the pressure to chase every new innovation.

Current multimodal models shoehorn visual data into a 1D text-based sequence. True spatial intelligence is different. It requires a native 3D/4D representation to understand a world governed by physics, not just human-generated language. This is a foundational architectural shift, not an extension of LLMs.