AI models show impressive performance on evaluation benchmarks but underwhelm in real-world applications. This gap exists because researchers, focused on evals, create reinforcement learning (RL) environments that mirror test tasks. This leads to narrow intelligence that doesn't generalize, a form of human-driven reward hacking.

There's a significant gap between AI performance in simulated benchmarks and in the real world. Despite scoring highly on evaluations, AIs in real deployments make "silly mistakes that no human would ever dream of doing," suggesting that current benchmarks don't capture the messiness and unpredictability of reality.

For physical AI systems like robots, data quality hinges on diversity, not just quantity. A robot trained to make a bed in one specific lighting condition may fail completely if the lighting changes or the bed is moved. This brittleness highlights a key challenge: training data must capture a wide variety of contexts and edge cases to enable real-world generalization.

Today's AI systems exhibit "jagged intelligence"—strong performance on many tasks but inconsistent reliability on others. This prevents full job replacement because being 95% effective is insufficient when the remaining 5% involves crucial edge cases, judgment, and discretion that still require human oversight.

The most fundamental challenge in AI today is not scale or architecture, but the fact that models generalize dramatically worse than humans. Solving this sample efficiency and robustness problem is the true key to unlocking the next level of AI capabilities and real-world impact.

AI struggles to provide truly useful, serendipitous recommendations because it lacks any understanding of the real world. It excels at predicting the next word or pixel based on its training data, but it can't grasp concepts like gravity or deep user intent, a prerequisite for truly personalized suggestions.

Karpathy warns that training AIs on synthetically generated data is dangerous due to "model collapse." An AI's output, while seemingly reasonable case-by-case, occupies a tiny, low-entropy manifold of the possible solution space. Continual training on this collapsed distribution causes the model to become worse and less diverse over time.

AI systems often collapse because they are built on the flawed assumption that humans are logical and society is static. Real-world failures, from Soviet economic planning to modern systems, stem from an inability to model human behavior, data manipulation, and unexpected events.

The central challenge for current AI is not merely sample efficiency but a more profound failure to generalize. Models generalize 'dramatically worse than people,' which is the root cause of their brittleness, inability to learn from nuanced instruction, and unreliability compared to human intelligence. Solving this is the key to the next paradigm.