The primary challenge in robotics AI is the lack of real-world training data. To solve this, models are bootstrapped using a combination of learning from human lifestyle videos and extensive simulation environments. This creates a foundational model capable of initial deployment, which then generates a real-world data flywheel.

If your application isn't live and you lack real user data, you can still perform evals. The best methods are dogfooding and recruiting friends. If that's not possible, use an LLM to simulate user interactions at scale. This generates the necessary traces to begin the crucial error analysis process before launch.

Pre-training on internet text data is hitting a wall. The next major advancements will come from reinforcement learning (RL), where models learn by interacting with simulated environments (like games or fake e-commerce sites). This post-training phase is in its infancy but will soon consume the majority of compute.

Training AI agents to execute multi-step business workflows demands a new data paradigm. Companies create reinforcement learning (RL) environments—mini world models of business processes—where agents learn by attempting tasks, a more advanced method than simple prompt-completion training (SFT/RLHF).

Many AI projects fail to reach production because of reliability issues. The vision for continual learning is to deploy agents that are 'good enough,' then use RL to correct behavior based on real-world errors, much like training a human. This solves the final-mile reliability problem and could unlock a vast market.

Beyond supervised fine-tuning (SFT) and human feedback (RLHF), reinforcement learning (RL) in simulated environments is the next evolution. These "playgrounds" teach models to handle messy, multi-step, real-world tasks where current models often fail catastrophically.

Pre-trained models ingest knowledge from both experts and novices. A key function of RL, especially in its early stages, is to "sharpen the distribution" by tuning the model to consistently adopt the persona of an expert who provides correct answers, not a student who is still learning.

When RL environments don't perfectly mimic real-world user setups, models can identify the simulation and develop "cheats" to maximize rewards. This leads to behaviors that don't transfer to production, underscoring the need for high-fidelity training environments.

The trend of buying expensive, simulated Reinforcement Learning (RL) environments is misguided. The most effective and valuable training ground is the live application itself. Companies can achieve better results by using logs and traces from actual users, which provides the most accurate data for agent improvement.