Models like Fable excel on benchmarks like Frontier Code because the underlying open-source repositories are well-tested and structured for external contributions. Most enterprise codebases lack these "deterministic feedback loops," meaning agentic performance in the real world is far worse than benchmarks suggest. The bottleneck isn't the model, it's the codebase's "agent readiness."

An AI model's operating environment—its "harness"—is now the primary driver of capability. Benchmarks show the same model achieves vastly different results in different harnesses, proving that the runtime, tools, and state management are as critical as the model's internal weights for achieving results.

The gap between benchmark scores and real-world performance suggests labs achieve high scores by distilling superior models or training for specific evals. This makes benchmarks a poor proxy for genuine capability, a skepticism that should be applied to all new model releases.

An AI coding agent's performance is driven more by its "harness"—the system for prompting, tool access, and context management—than the underlying foundation model. This orchestration layer is where products create their unique value and where the most critical engineering work lies.

The latest Arc AGI benchmark ditches static puzzles for interactive games with no instructions. This forces models to explore, learn rules, and adapt on the fly. It directly measures their ability to acquire new skills efficiently—a closer proxy for general intelligence than testing memorized reasoning patterns.

Performance comes from a "harness" surrounding the AI model, which includes curated data, tools, and rich context. This harness, which can be open and multi-model, is where the hard work lies—prepping the context layer so that a model's plan can execute efficiently.

The disconnect between AI's superhuman benchmark scores and its limited economic impact exists because many benchmarks test esoteric problems. The Arc AGI prize instead focuses on tasks that are easy for humans, testing an AI's ability to learn new concepts from few examples—a better proxy for general, applicable intelligence.

Google's new state-of-the-art Deep Research agents are still powered by the older Gemini 3.1 Pro model. Their significant performance improvements come entirely from 'harness upgrades' and additional inference techniques. This demonstrates that the systems, tools, and processes surrounding a model are now a primary driver of capability, not just the raw power of the base model itself.

Judging an AI's capability by its base model alone is misleading. Its effectiveness is significantly amplified by surrounding tooling and frameworks, like developer environments. A good tool harness can make a decent model outperform a superior model that lacks such support.