In a group of 100 experts training an AI, the top 10% will often drive the majority of the model's improvement. This creates a power law dynamic where the ability to source and identify this elite talent becomes a key competitive moat for AI labs and data providers.

Related Insights

AI startup Mercore's valuation quintupled to $10B by connecting AI labs with domain experts to train models. This reveals that the most critical bottleneck for advanced AI is not just data or compute, but reinforcement learning from highly skilled human feedback, creating a new "RL economy."

Early AI training involved simple preference tasks. Now, training frontier models requires PhDs and top professionals to perform complex, hours-long tasks like building entire websites or explaining nuanced cancer topics. The demand is for deep, specialized expertise, not just generalist labor.

LLMs have hit a wall by scraping nearly all available public data. The next phase of AI development and competitive differentiation will come from training models on high-quality, proprietary data generated by human experts. This creates a booming "data as a service" industry for companies like Micro One that recruit and manage these experts.

To move beyond general knowledge, AI firms are creating a new role: the "AI Trainer." These are not contractors but full-time employees, typically PhDs with deep domain expertise and a computer science interest, tasked with systematically improving model competence in specific fields like physics or mathematics.

While many believe AI will primarily help average performers become great, LinkedIn's experience shows the opposite. Their top talent were the first and most effective adopters of new AI tools, using them to become even more productive. This suggests AI may amplify existing talent disparities.

Multi-million dollar salaries for top AI researchers seem absurd, but they may be underpaid. These individuals aren't just employees; they are capital allocators. A single architectural decision can tie up or waste months of capacity on billion-dollar AI clusters, making their judgment incredibly valuable.

For years, access to compute was the primary bottleneck in AI development. Now, as public web data is largely exhausted, the limiting factor is access to high-quality, proprietary data from enterprises and human experts. This shifts the focus from building massive infrastructure to forming data partnerships and expertise.

Mercore's $500M revenue in 17 months highlights a shift in AI training. The focus is moving from low-paid data labelers to a marketplace of elite experts like doctors and lawyers providing high-quality, nuanced data. This creates a new, lucrative gig economy for top-tier professionals.

For cutting-edge AI problems, innate curiosity and learning speed ("velocity") are more important than existing domain knowledge. Echoing Karpathy, a candidate with a track record of diving deep into complex topics, regardless of field, will outperform a skilled but less-driven specialist.

Contrary to the belief that distribution is the new moat, the crucial differentiator in AI is talent. Building a truly exceptional AI product is incredibly nuanced and complex, requiring a rare skill set. The scarcity of people who can build off models in an intelligent, tasteful way is the real technological moat, not just access to data or customers.

A Few Elite Human Experts Drive Most of an AI Model's Performance Gains | RiffOn