Andrej Karpathy's 'Software 2.0' framework posits that AI automates tasks that are easily *verifiable*. This explains the 'jagged frontier' of AI progress: fields like math and code, where correctness is verifiable, advance rapidly. In contrast, creative and strategic tasks, where success is subjective and hard to verify, lag significantly behind.

AI models excel in domains with discrete, quantifiable outcomes like coding or chess. However, they struggle with most knowledge work, which is often "unverifiable" and lacks a single correct answer for reinforcement learning models to train on. This distinction explains AI's current limitations in many professional roles.

In domains like coding and math where correctness is automatically verifiable, AI can move beyond imitating humans (RLHF). Using pure reinforcement learning, or "experiential learning," models learn via self-play and can discover novel, superhuman strategies similar to AlphaGo's Move 37.

It's a misconception that Reinforcement Learning's power is limited to domains with clear, verifiable rewards. Geoffrey Irving points out that frontier models use RL to improve on fuzzy, unverifiable tasks, like giving troubleshooting advice from a photo of a lab setup, proving the technique's much broader effectiveness.

Axiom's success on the Putnam exam suggests verified generation offers significant performance gains and sample efficiency. This allows a focused startup with less compute and data to outperform generalist frontier lab models on complex, superhuman reasoning tasks.

The "one-time subsidy" of high-quality internet text is largely exhausted. The future of data for training models lies in creating reinforcement learning (RL) "gyms" where agents work on hard, verifiable problems (e.g., coding, math). The environment and the agent's progress become the new data source for recursive self-improvement.

Demis Hassabis identifies a key obstacle for AGI. Unlike in math or games where answers can be verified, the messy real world lacks clear success metrics. This makes it difficult for AI systems to use self-improvement loops, limiting their ability to learn and adapt outside of highly structured domains.

Unlike math or code with cheap, fast rewards, clinically valuable biology problems lack easily verifiable ground truths. This makes it difficult to create the rapid reinforcement learning loops that drive explosive AI progress in other fields.

Verifiability alone doesn't explain AI's rapid progress in math and coding. The key factor is 'grindability'—the ability to run thousands of parallel, containerized, and deterministic simulations. This allows for efficient credit assignment and learning, a luxury not available in domains like e-commerce or business strategy, which are constrained by real-world interactions and bot detectors.