It's surprising that AI models trained on general data can accurately predict rare events like hurricanes. The reason is that the physical world is "forgiving"; extreme phenomena are governed by strong physical structures and signatures that AI can learn effectively, even from a limited number of examples.

Goodfire's research on 'neural geometry' reveals that concepts inside models have distinct, low-dimensional shapes (manifolds). For example, numbers form a helix and temperature forms a spiral arc. Understanding these shapes allows for more precise and effective interventions, moving beyond linear vector manipulations.

Counter-intuitively, successful weather and climate AI models are not trained on long-term data. They are trained to predict only the next six hours autoregressively. This surprisingly generalizes to stable rollouts predicting weather patterns for hundreds or thousands of steps into the future, enabling long-term forecasting from short-term training.

Traditional weather forecasting requires massive supercomputers, limiting access to large agencies. New AI models are tens of thousands of times faster and can run on a single consumer-grade GPU. This democratizes high-fidelity weather modeling for smaller agencies and nations, especially in the global south.

Large Language Models are limited because they lack an understanding of the physical world. The next evolution is 'World Models'—AI trained on real-world sensory data to understand physics, space, and context. This is the foundational technology required to unlock physical AI like advanced robotics.

Most AI weather models project the Earth onto a flat rectangle, causing simulations to become unstable and "blow up" over long periods. By incorporating the planet's spherical geometry using Fourier Neural Operators, models like ForecastNet remain stable for long-term climate rollouts, effectively becoming climate models.

While dominant in 1D language tasks, the quadratic complexity of transformers makes them computationally infeasible for high-resolution 3D and 4D physical simulations. An industrial-scale grid can represent a context length in the "hundreds of billions to even a trillion," far beyond what any transformer can handle.

PINs, which solve PDEs from scratch using only physics constraints, often fail on complex, time-dependent problems due to difficult optimization landscapes. Neural operators overcome this by using a data-driven, supervised learning approach, learning from existing solutions before applying physics constraints, making them more robust.

While traditional hybrid models still require solving differential equations during use, Physics-Informed Neural Networks (PINNs) learn the final outcome directly. They are trained to obey physical laws but don't need the equations for inference. This provides a significant speed advantage for computationally expensive systems like chromatography, making them more suitable for real-time applications.