AI models show impressive performance on evaluation benchmarks but underwhelm in real-world applications. This gap exists because researchers, focused on evals, create reinforcement learning (RL) environments that mirror test tasks. This leads to narrow intelligence that doesn't generalize, a form of human-driven reward hacking.

There's a significant gap between AI performance in simulated benchmarks and in the real world. Despite scoring highly on evaluations, AIs in real deployments make "silly mistakes that no human would ever dream of doing," suggesting that current benchmarks don't capture the messiness and unpredictability of reality.

Applications relying solely on generic, off-the-shelf foundation models will eventually hit a performance ceiling. Achieving superior, order-of-magnitude better results for specific workflows requires building a "micro model" through custom data labeling, fine-tuning, and creating a unique reasoning layer to create a defensible product.

The most significant gap in AI research is its focus on academic evaluations instead of tasks customers value, like medical diagnosis or legal drafting. The solution is using real-world experts to define benchmarks that measure performance on economically relevant work.

Frontier AI models exhibit 'jagged intelligence,' excelling at complex tasks like PhD-level science but failing at simple ones like reading a clock. This inconsistency means businesses cannot trust external benchmarks and must create their own internal evaluations based on specific company workflows.

Just as standardized tests fail to capture a student's full potential, AI benchmarks often don't reflect real-world performance. The true value comes from the 'last mile' ingenuity of productization and workflow integration, not just raw model scores, which can be misleading.

AI performance on clean benchmarks overestimates real-world utility. In practice, tasks are "messy"—involving collaboration, large codebases, and adversarial situations—which current AIs handle poorly. This gap explains why productivity gains lag behind benchmark scores.

Alex Karp argues that an AI's high score on a single benchmark is irrelevant for enterprise adoption. Real institutions require passing thousands of consecutive, differentiated tests. An AI model that is brilliant at one task but fails at the 50th in a complex sequence is effectively useless.

The rapid improvement of AI models is maxing out industry-standard benchmarks for tasks like software engineering. To truly understand AI's impact and capability, companies must develop their own evaluation systems tailored to their specific workflows, rather than waiting for external studies.