We scan new podcasts and send you the top 5 insights daily.
Frontier models rapidly improve in fields like coding and mathematics because correctness can be automatically validated and hill-climbed. In open-ended scientific and business domains without ground-truth answers, long-horizon agents struggle due to slow real-world feedback loops.
While AI excels at writing software—a domain with clear, verifiable outcomes (it compiles or it doesn't)—its application in fields like drug discovery is limited. Progress stalls where outcomes aren't easily and quickly verifiable, requiring complex, expensive, real-world testing.
Reinforcement learning with verifiable rewards allows models to achieve super-human mathematical and coding performance rapidly, because outcomes can be objectively checked in simulated environments. In contrast, progress in non-verifiable domains advances much slower because improvement remains tethered to human data generation bottlenecks. The resulting jagged intelligence frontier creates super-intelligence within narrow formal environments while general capabilities lag behind.
Andrej Karpathy's 'Software 2.0' framework posits that AI automates tasks that are easily *verifiable*. This explains the 'jagged frontier' of AI progress: fields like math and code, where correctness is verifiable, advance rapidly. In contrast, creative and strategic tasks, where success is subjective and hard to verify, lag significantly behind.
AI models excel in domains with discrete, quantifiable outcomes like coding or chess. However, they struggle with most knowledge work, which is often "unverifiable" and lacks a single correct answer for reinforcement learning models to train on. This distinction explains AI's current limitations in many professional roles.
Judgment Labs CEO Alex Shan argues that AI agents will first dominate domains with easily verifiable results, like coding, where a solution's correctness can be quickly checked. Progress will be slower in non-verifiable fields like law or complex drug discovery, where feedback loops are long and ambiguous.
Friedberg and Sacks explain that AI advances much faster in mathematics and software engineering because their reasoning loops are completely testable in software. Unlike biology or logistics, these disciplines require no physical lab testing or human-in-the-loop validation, enabling continuous automated reinforcement learning.
Demis Hassabis identifies a key obstacle for AGI. Unlike in math or games where answers can be verified, the messy real world lacks clear success metrics. This makes it difficult for AI systems to use self-improvement loops, limiting their ability to learn and adapt outside of highly structured domains.
AI models improve dramatically in domains with objective feedback, like coding (unit tests) or science (lab results). Progress is slower in subjective fields like creative writing where feedback is opinion-based, explaining the uneven impact of AI across different types of knowledge work.
The path to AI self-improvement isn't uniform. It is happening first in software engineering and AI research because these fields have cheap, fast, and verifiable feedback (e.g., unit tests). This capability won't automatically transfer to domains like biology until similar closed-loop systems are built.
AI excels at tasks where it can make small attempts and get fast, clear feedback ('hill climbing'). Rogue deployments require long-horizon strategic planning with no easy feedback, a domain where agents are currently very weak.