ZAI's new model demonstrates that significant performance gains, nearing state-of-the-art in specialized areas, can be achieved by intensely scaling reinforcement learning on a mid-sized base model. This challenges the prevailing narrative that ever-larger parameter counts are the only path to frontier capabilities.

While RL may not provide perfect cross-domain reasoning (e.g., math to code), its key contribution is teaching models 'horizon generalization.' This is the ability to use more tokens productively over a longer period to make progress on a complex task. This meta-skill is a primary driver of recent capability improvements and appears to be doubling every three months.

Reinforcement learning achieves superhuman results not by inventing alien concepts, but by surfacing and combining rare behaviors that are already possible within a model's vast pre-trained distribution. The goal of pre-training is to make this search for novel solutions more efficient and less random.

Pre-training on internet text data is hitting a wall. The next major advancements will come from reinforcement learning (RL), where models learn by interacting with simulated environments (like games or fake e-commerce sites). This post-training phase is in its infancy but will soon consume the majority of compute.

AI labs like Anthropic find that mid-tier models can be trained with reinforcement learning to outperform their largest, most expensive models in just a few months, accelerating the pace of capability improvements.

Pre-trained models ingest knowledge from both experts and novices. A key function of RL, especially in its early stages, is to "sharpen the distribution" by tuning the model to consistently adopt the persona of an expert who provides correct answers, not a student who is still learning.

Goodfire's research operates on the premise that post-training processes like RL don't teach models fundamentally new capabilities. Instead, they primarily make low-likelihood events and behaviors already present from pre-training more probable, essentially shaping the model's existing knowledge.

AI development is inefficiently split into pre-training (optimizing for compression) and RL (optimizing for tasks), where RL often invalidates pre-training metrics. Combining these into a unified, end-to-end learning algorithm focused on final outcomes could yield an order-of-magnitude improvement in training efficiency.

Reinforcement Learning is effective because it applies a few, high-signal bits of feedback ('correct' or 'incorrect') to an already capable model. Unlike Supervised Fine-Tuning (SFT), which is noisy and tries to match every token, RL isolates the most crucial learning signal. This allows for efficient tweaking of a model's policy without reteaching it everything.