Early LLM benchmarks like MMLU tested question-answering, a now-saturated capability. Today, leading labs like Anthropic evaluate models on agentic tasks involving multi-step reasoning and tool use, reflecting the shift in AI applications from search replacement to automated workflows.

AI models show impressive performance on evaluation benchmarks but underwhelm in real-world applications. This gap exists because researchers, focused on evals, create reinforcement learning (RL) environments that mirror test tasks. This leads to narrow intelligence that doesn't generalize, a form of human-driven reward hacking.

There's a significant gap between AI performance in simulated benchmarks and in the real world. Despite scoring highly on evaluations, AIs in real deployments make "silly mistakes that no human would ever dream of doing," suggesting that current benchmarks don't capture the messiness and unpredictability of reality.

Current AI models resemble a student who grinds 10,000 hours on a narrow task. They achieve superhuman performance on benchmarks but lack the broad, adaptable intelligence of someone with less specific training but better general reasoning. This explains the gap between eval scores and real-world utility.

Issues like 'saturation' and 'maxing' reveal a fundamental flaw: benchmarks test narrow, siloed abilities ('Task AGI'). They fail to measure an AI's capacity to combine skills to solve multi-step problems, which is the true bottleneck preventing real-world agentic performance and the next frontier of AI.

The Stanford AI Index reveals a "jagged frontier" where advanced models achieve superhuman performance on complex tasks like the International Mathematical Olympiad, yet fail at simple, common-sense activities like reading an analog clock. This highlights their lack of real-world grounding and the need for more holistic "world models."

While AI models excel at gathering and synthesizing information ('knowing'), they are not yet reliable at executing actions in the real world ('doing'). True agentic systems require bridging this gap by adding crucial layers of validation and human intervention to ensure tasks are performed correctly and safely.

Many AI benchmarks focus on arbitrary, synthetic tasks (like a "pelican riding a bicycle" SVG test) that don't reflect real user workflows. This creates a disconnect where models top leaderboards but fail at practical jobs. True value is measured by observing users getting their work done.

Seemingly simple user requests require a complex sequence of reasoning, tool use, and contextual understanding that is absent from internet training data. AI must be explicitly taught the implicit logic of how a human assistant would research preferences, evaluate options, and use various tools.