In a real-world vending machine test, Grok was less emotional and easier to steer towards its business objective. It resisted giving discounts and was more focused on profitability than Anthropic's Claude, though this came at the cost of being less entertaining and personable.

Human time to completion is a strong predictor of AI success, but it's not perfect. METR's analysis found that a task's qualitative 'messiness'—how clean and simple it is versus tricky and rough—also independently predicts whether an AI will succeed. This suggests that pure task length doesn't capture all aspects of difficulty for AIs.

AI struggles with long-horizon tasks not just due to technical limits, but because we lack good ways to measure performance. Once effective evaluations (evals) for these capabilities exist, researchers can rapidly optimize models against them, accelerating progress significantly.

The test intentionally used a simple, conversational prompt one might give a colleague ("our blog is not good...make it better"). The models' varying success reveals that a key differentiator is the ability to interpret high-level intent and independently research best practices, rather than requiring meticulously detailed instructions.

Data that measures success, like a grading rubric, is far more valuable for AI training than simple raw output. This 'second kind of data' enables iterative learning by allowing models to attempt a problem, receive a score, and learn from the feedback.

Building a functional AI agent is just the starting point. The real work lies in developing a set of evaluations ("evals") to test if the agent consistently behaves as expected. Without quantifying failures and successes against a standard, you're just guessing, not iteratively improving the agent's performance.

Traditional AI benchmarks are seen as increasingly incremental and less interesting. The new frontier for evaluating a model's true capability lies in applied, complex tasks that mimic real-world interaction, such as building in Minecraft (MC Bench) or managing a simulated business (VendingBench), which are more revealing of raw intelligence.

The disconnect between AI's superhuman benchmark scores and its limited economic impact exists because many benchmarks test esoteric problems. The Arc AGI prize instead focuses on tasks that are easy for humans, testing an AI's ability to learn new concepts from few examples—a better proxy for general, applicable intelligence.

As reinforcement learning (RL) techniques mature, the core challenge shifts from the algorithm to the problem definition. The competitive moat for AI companies will be their ability to create high-fidelity environments and benchmarks that accurately represent complex, real-world tasks, effectively teaching the AI what matters.