Unlike traditional neural networks which require fixed-resolution inputs (e.g., pixels), neural operators model data as continuous functions. This allows them to "zoom in" and make predictions at resolutions higher than the training data, a crucial capability for multi-scale physical phenomena like weather patterns.
PINs, which solve PDEs from scratch using only physics constraints, often fail on complex, time-dependent problems due to difficult optimization landscapes. Neural operators overcome this by using a data-driven, supervised learning approach, learning from existing solutions before applying physics constraints, making them more robust.
Counter-intuitively, successful weather and climate AI models are not trained on long-term data. They are trained to predict only the next six hours autoregressively. This surprisingly generalizes to stable rollouts predicting weather patterns for hundreds or thousands of steps into the future, enabling long-term forecasting from short-term training.
Fourier transforms offer a sweet spot for modeling physical systems. They capture non-local interactions (like global weather patterns) with quasi-linear complexity, avoiding the untenable quadratic complexity of transformers when applied to high-resolution 3D or 4D data. This makes large-scale physical simulation with AI feasible.
It's surprising that AI models trained on general data can accurately predict rare events like hurricanes. The reason is that the physical world is "forgiving"; extreme phenomena are governed by strong physical structures and signatures that AI can learn effectively, even from a limited number of examples.
A 'one size fits all' approach to AI regulation is flawed because it often equates all of AI with language models. AI for science has fundamentally different applications, risks, and benefits, such as discovering new materials or medicines. Regulatory frameworks must be nuanced to avoid stifling scientific progress.
While dominant in 1D language tasks, the quadratic complexity of transformers makes them computationally infeasible for high-resolution 3D and 4D physical simulations. An industrial-scale grid can represent a context length in the "hundreds of billions to even a trillion," far beyond what any transformer can handle.
Traditional weather forecasting requires massive supercomputers, limiting access to large agencies. New AI models are tens of thousands of times faster and can run on a single consumer-grade GPU. This democratizes high-fidelity weather modeling for smaller agencies and nations, especially in the global south.
Most AI weather models project the Earth onto a flat rectangle, causing simulations to become unstable and "blow up" over long periods. By incorporating the planet's spherical geometry using Fourier Neural Operators, models like ForecastNet remain stable for long-term climate rollouts, effectively becoming climate models.
For safety-critical applications like controlling a nuclear reactor, AI models need provable guarantees. TorchLean allows developers to formally verify properties of neural networks, such as bounding how much the output can change given an input perturbation, ensuring stability and safety in control loops.
Current AI models for science are narrow surrogates for specific tasks. The grand vision is to build a foundation model for physics that understands a wide range of coupled, multi-physics phenomena. This single model could be used for simulation, inverse design, and control across many scientific and engineering domains.