Early AI training involved simple preference tasks. Now, training frontier models requires PhDs and top professionals to perform complex, hours-long tasks like building entire websites or explaining nuanced cancer topics. The demand is for deep, specialized expertise, not just generalist labor.

A consortium including leaders from Google and DeepMind has defined AGI as matching the cognitive versatility of a "well-educated adult" across 10 domains. This new framework moves beyond abstract debate, showing a concrete 30-point leap in AGI score from GPT-4 (27%) to a projected GPT-5 (57%).

Formal AI competency frameworks are still emerging. In their place, innovative companies are assessing employee AI skills with concrete, activity-based targets like "build three custom GPTs for your role" or completing specific certifications, directly linking these achievements to performance reviews.

A benchmark testing AI agents against paid freelance jobs found the best performers could only autonomously complete 2.5% of the work. This provides a crucial reality check, showing that while AI excels at discrete tasks, full job automation by general-purpose agents is still far from reality.

OpenAI's new GDPVal framework evaluates AI on real-world knowledge work. It found frontier models produce work rated equal to or better than human experts nearly 50% of the time, while being 100 times faster and cheaper. This provides a direct measure of impending economic transformation.

Current AI models resemble a student who grinds 10,000 hours on a narrow task. They achieve superhuman performance on benchmarks but lack the broad, adaptable intelligence of someone with less specific training but better general reasoning. This explains the gap between eval scores and real-world utility.

The primary bottleneck in improving AI is no longer data or compute, but the creation of 'evals'—tests that measure a model's capabilities. These evals act as product requirement documents (PRDs) for researchers, defining what success looks like and guiding the training process.

As reinforcement learning (RL) techniques mature, the core challenge shifts from the algorithm to the problem definition. The competitive moat for AI companies will be their ability to create high-fidelity environments and benchmarks that accurately represent complex, real-world tasks, effectively teaching the AI what matters.

Standardized AI benchmarks are saturated and becoming less relevant for real-world use cases. The true measure of a model's improvement is now found in custom, internal evaluations (evals) created by application-layer companies. Progress for a legal AI tool, for example, is a more meaningful indicator than a generic test score.