When AI models achieve superhuman performance on specific benchmarks like coding challenges, it doesn't solve real-world problems. This is because we implicitly optimize for the benchmark itself, creating "peaky" performance rather than broad, generalizable intelligence.

Language is a human-optimized construct, but the visual world is not. It contains a "fat tail" of chaotic scenes that are harder for models to learn, explaining why vision capabilities today resemble natural language processing from the GPT-3 era.

There's a significant gap between AI performance in simulated benchmarks and in the real world. Despite scoring highly on evaluations, AIs in real deployments make "silly mistakes that no human would ever dream of doing," suggesting that current benchmarks don't capture the messiness and unpredictability of reality.

Current AI benchmarks have become targets for competition, an example of Goodhart's Law. Models are optimized to top leaderboards rather than develop the general capabilities the benchmarks were designed to measure, creating a false sense of progress and failing to predict real-world performance.

In the "Blueprint" benchmark, models were asked to create a floor plan from 20 interior apartment photos. They had to reason about 3D space and stitch together different views. No model performed statistically better than random chance, highlighting a major, quantified deficit in the spatial intelligence of current multimodal systems.

Issues like 'saturation' and 'maxing' reveal a fundamental flaw: benchmarks test narrow, siloed abilities ('Task AGI'). They fail to measure an AI's capacity to combine skills to solve multi-step problems, which is the true bottleneck preventing real-world agentic performance and the next frontier of AI.

The tests for AI image models have shifted from generating novel concepts ('astronaut on a horse') to solving logical inversions ('horse on an astronaut') and subtle details ('a completely full wine glass'). This progression demonstrates the 'moving the goalposts' phenomenon in AI, where humans continuously invent harder tests as technology improves.

AI performance on clean benchmarks overestimates real-world utility. In practice, tasks are "messy"—involving collaboration, large codebases, and adversarial situations—which current AIs handle poorly. This gap explains why productivity gains lag behind benchmark scores.

Don't trust academic benchmarks. Labs often "hill climb" or game them for marketing purposes, which doesn't translate to real-world capability. Furthermore, many of these benchmarks contain incorrect answers and messy data, making them an unreliable measure of true AI advancement.