In a 2018 interview, OpenAI's Greg Brockman described their foundational training method: ingesting thousands of books with the sole task of predicting the next word. This simple predictive objective was the key that unlocked complex, generalizable language understanding in their models.

A key property of Recursive Language Models (RLMs) is their ability to decompose a large, complex task that is out-of-distribution (OOD) for the model. The harness breaks it into a series of smaller sub-problems, each of which is locally in-distribution, ensuring more reliable performance at each step.

While RL may not provide perfect cross-domain reasoning (e.g., math to code), its key contribution is teaching models 'horizon generalization.' This is the ability to use more tokens productively over a longer period to make progress on a complex task. This meta-skill is a primary driver of recent capability improvements and appears to be doubling every three months.

The structured, hierarchical nature of code (functions, libraries) provides a powerful training signal for AI models. This helps them infer structural cues applicable to broader reasoning and planning tasks, far beyond just code generation.

Reinforcement learning achieves superhuman results not by inventing alien concepts, but by surfacing and combining rare behaviors that are already possible within a model's vast pre-trained distribution. The goal of pre-training is to make this search for novel solutions more efficient and less random.

The argument that LLMs are just "stochastic parrots" is outdated. Current frontier models are trained via Reinforcement Learning, where the signal is not "did you predict the right token?" but "did you get the right answer?" This is based on complex, often qualitative criteria, pushing models beyond simple statistical correlation.

Training an AI model in a complex, non-coding environment—requiring it to use tools, parse documents, and follow instructions—unexpectedly improves its coding abilities. This suggests that teaching generalized reasoning and tool-use is more effective than narrow, task-specific training.

A harness's design is an opinionated program that shapes how a model approaches a problem. A well-designed harness, like an RLM, can dramatically increase a model's generalization capabilities by providing a structural prior that helps it solve tasks more efficiently.

The model's training used "response only masking," where it only learns from the response part of the training data. This method forces the model to first generate a structured "chain of thought" before producing a final answer, directly embedding a systematic problem-solving process into its behavior.