Startups and major labs are focusing on "world models," which simulate physical reality, cause, and effect. This is seen as the necessary step beyond text-based LLMs to create agents that can truly understand and interact with the physical world, a key step towards AGI.

Foundation models can't be trained for physics using existing literature because the data is too noisy and lacks published negative results. A physical lab is needed to generate clean data and capture the learning signal from failed experiments, which is a core thesis for Periodic Labs.

To break the data bottleneck in AI protein engineering, companies now generate massive synthetic datasets. By creating novel "synthetic epitopes" and measuring their binding, they can produce thousands of validated positive and negative training examples in a single experiment, massively accelerating model development.

Synthetic data serves as an efficient first step for training specialized AI, particularly when a larger model teaches a smaller one. However, it is insufficient on its own. The final, crucial stage always requires expensive "human signal"—feedback from subject matter experts—to achieve true performance.

Advanced model training is not just about scraping the web. It's a multi-stage process that starts with massive web data, is refined by human-created examples and ratings (SFT), and is then scaled using reinforcement learning on data generated by the model itself. This synthetic data loop is now a critical component.

Unlike protein folding, which benefited from the CASP competition's experimental ground truth data, materials science lacks large-scale, high-quality experimental datasets. Existing data often comes from low-fidelity simulations, meaning even the best AI models are trained on imperfect information, hindering a major breakthrough.

The future of valuable AI lies not in models trained on the abundant public internet, but in those built on scarce, proprietary data. For fields like robotics and biology, this data doesn't exist to be scraped; it must be actively created, making the data generation process itself the key competitive moat.

The push toward physical AI and spatial intelligence is primarily a strategy to overcome data scarcity for training general models. By creating simulated 3D environments, researchers can generate the novel, complex data that is currently unavailable but crucial for advancing AI into the real world.

Early AI models advanced by scraping web text and code. The next revolution, especially in "AI for science," requires overcoming a major hurdle: consolidating and formatting the world's vast but fragmented scientific data across disciplines like chemistry and materials science for model training.