Cursor achieved performance competitive with OpenAI's and Anthropic's best models not by training from scratch, but by applying superior reinforcement learning to an existing base model. This demonstrates a viable, data-driven path for smaller companies to compete on model quality without massive upfront compute.

Reinforcement learning achieves superhuman results not by inventing alien concepts, but by surfacing and combining rare behaviors that are already possible within a model's vast pre-trained distribution. The goal of pre-training is to make this search for novel solutions more efficient and less random.

The 'bigger is better' narrative is breaking down. For well-defined, structured tasks like coding and math, small models (e.g., 3 billion parameters) are now matching the performance of frontier models. This enables powerful, specialized AI to run on modest local hardware.

Pre-trained models ingest knowledge from both experts and novices. A key function of RL, especially in its early stages, is to "sharpen the distribution" by tuning the model to consistently adopt the persona of an expert who provides correct answers, not a student who is still learning.

Performance on knowledge-intensive benchmarks correlates strongly with an MoE model's total parameter count, not its active parameter count. With leading models like Kimi K2 reportedly using only ~3% active parameters, this suggests there is significant room to increase sparsity and efficiency without degrading factual recall.

Rather than achieving general intelligence through abstract reasoning, AI models improve by repeatedly identifying specific failures (like trick questions) and adding those scenarios into new training rounds. This "patching" approach, though seemingly inefficient, proved successful for self-driving cars and may be a viable path for language models.

While frontier science may require massive models, most knowledge work relies on persistence and backtracking. As smaller models improve these behaviors, the optimal size for tasks like coding might be reached far sooner than expected, making open-source models highly competitive for a vast segment of the economy.

Microsoft's research found that training smaller models on high-quality, synthetic, and carefully filtered data produces better results than training larger models on unfiltered web data. Data quality and curation, not just model size, are the new drivers of performance.

Like human experts, advanced AI models improve their answers the more time they spend on a problem. This 'inference scaling' means short evaluations may fail to capture a model's true capabilities, as performance continues to increase with more computation, making it difficult to establish a performance ceiling.