Static benchmarks are easily gamed. Dynamic environments like the game Diplomacy force models to negotiate, strategize, and even lie, offering a richer, more realistic evaluation of their capabilities beyond pure performance metrics like reasoning or coding.

Standard AI benchmarks are an engineering tool for measuring performance. A more scientific approach, borrowed from cognitive psychology, uses targeted experiments. By designing problems where specific patterns of success and failure are diagnostic, researchers can uncover the underlying mechanisms and principles of an AI system, yielding deeper insights than a simple score.

OpenAI's evals team is looking beyond current benchmarks that test self-contained, hour-long tasks. They are calling for new evaluations that measure performance on problems that would take top engineers weeks or months to solve, such as creating entire products end-to-end. This signals a major increase in the complexity and ambition expected from future AI benchmarks.

AI struggles with long-horizon tasks not just due to technical limits, but because we lack good ways to measure performance. Once effective evaluations (evals) for these capabilities exist, researchers can rapidly optimize models against them, accelerating progress significantly.

Issues like 'saturation' and 'maxing' reveal a fundamental flaw: benchmarks test narrow, siloed abilities ('Task AGI'). They fail to measure an AI's capacity to combine skills to solve multi-step problems, which is the true bottleneck preventing real-world agentic performance and the next frontier of AI.

The latest Arc AGI benchmark ditches static puzzles for interactive games with no instructions. This forces models to explore, learn rules, and adapt on the fly. It directly measures their ability to acquire new skills efficiently—a closer proxy for general intelligence than testing memorized reasoning patterns.

Traditional AI benchmarks are seen as increasingly incremental and less interesting. The new frontier for evaluating a model's true capability lies in applied, complex tasks that mimic real-world interaction, such as building in Minecraft (MC Bench) or managing a simulated business (VendingBench), which are more revealing of raw intelligence.

The disconnect between AI's superhuman benchmark scores and its limited economic impact exists because many benchmarks test esoteric problems. The Arc AGI prize instead focuses on tasks that are easy for humans, testing an AI's ability to learn new concepts from few examples—a better proxy for general, applicable intelligence.

As reinforcement learning (RL) techniques mature, the core challenge shifts from the algorithm to the problem definition. The competitive moat for AI companies will be their ability to create high-fidelity environments and benchmarks that accurately represent complex, real-world tasks, effectively teaching the AI what matters.