AI models are trained to find the most probable answer, reflecting the average of their data. Truly great, tasteful work is often unique and statistically unlikely, a quality that current models, which regress to the mean, struggle to produce. They can solve PhD-level math but fail at creative tasks like writing a good tweet.

When AI models achieve superhuman performance on specific benchmarks like coding challenges, it doesn't solve real-world problems. This is because we implicitly optimize for the benchmark itself, creating "peaky" performance rather than broad, generalizable intelligence.

Benchmarking revealed no strong correlation between a model's general intelligence and its tendency to hallucinate. This suggests that a model's "honesty" is a distinct characteristic shaped by its post-training recipe, not just a byproduct of having more knowledge.

Aidan Smith says models all talk the same way because they train on similar traces and good rollouts. If algorithms let models learn as much from as little data as humans, they can make inferences in stranger, more creative ways. That could reduce model homogenization.

Progress towards AGI is not a smooth climb. Models exhibit "spikiness"—they can perform at a world-class level on one narrow domain but degrade to a "bad high school student" with slight perturbations. This non-intuitive generalization makes their capabilities uneven and unpredictable.

Current AI models resemble a student who grinds 10,000 hours on a narrow task. They achieve superhuman performance on benchmarks but lack the broad, adaptable intelligence of someone with less specific training but better general reasoning. This explains the gap between eval scores and real-world utility.

Newer LLMs exhibit a more homogenized writing style than earlier versions like GPT-3. This is due to "style burn-in," where training on outputs from previous generations reinforces a specific, often less creative, tone. The model’s style becomes path-dependent, losing the raw variety of its original training data.

Sam Altman acknowledged that models are becoming "spiky," with capabilities improving unevenly. OpenAI intentionally prioritized making GPT-5.2 excel at reasoning and coding, which led to a degradation in its creative writing and prose. This highlights the trade-offs inherent in current model training.

Instead of only analyzing a fully trained model, "intentional design" seeks to control what a model learns during training. The goal is to shape the loss landscape to produce desired behaviors and generalizations from the outset, moving from archaeology to architecture.