To increase developer adoption, OpenAI intentionally trained its models on specific behavioral characteristics, not just coding accuracy. These 'personality' traits include communication (explaining its steps), planning, and self-checking, mirroring best practices of human software engineers to make the AI a more trustworthy pair programmer.

Minimax builds both foundation models and user-facing applications in-house. This structure enables research and engineering teams to work side-by-side, getting direct feedback from internal developers to rapidly identify and address model weaknesses, ensuring models meet real-world needs.

While RL is compute-intensive for the amount of signal it extracts, this is its core economic advantage. It allows labs to trade cheap, abundant compute for expensive, scarce human expertise. RL effectively amplifies the value of small, high-quality human-generated datasets, which is crucial when expertise is the bottleneck.

The frontier of AI training is moving beyond humans ranking model outputs (RLHF). Now, high-skilled experts create detailed success criteria (like rubrics or unit tests), which an AI then uses to provide feedback to the main model at scale, a process called RLAIF.

Reinforcement Learning with Human Feedback (RLHF) is a popular term, but it's just one method. The core concept is reinforcing desired model behavior using various signals. These can include AI feedback (RLAIF), where another AI judges the output, or verifiable rewards, like checking if a model's answer to a math problem is correct.

Unlike coding with its verifiable unit tests, complex legal work lacks a binary success metric. Harvey addresses this reinforcement learning challenge by treating senior partner feedback and edits as the "reward function," mirroring how quality is judged in the real world. The ultimate verification is long-term success, like a merger avoiding future litigation.

Instead of a single "think then act" cycle, Minimax trains its M2 model to repeatedly pause and rethink after receiving feedback from the environment. This iterative "interleaved thinking" approach improves robustness and performance on long-horizon tasks where tool responses or conditions are unpredictable.

When determining what data an RL model should consider, resist including every available feature. Instead, observe how experienced human decision-makers reason about the problem. Their simplified mental models reveal the core signals that truly drive outcomes, leading to more stable, faster-learning, and more interpretable AI systems.

Instead of relying on digital proxies like code graders, Periodic Labs uses real-world lab experiments as the ultimate reward function. Nature itself becomes the reinforcement learning environment, ensuring the AI is optimized against physical reality, not flawed simulations.