Formal proof systems like Lean provide a unique training ground for LLMs. Unlike natural language reasoning, a proof's correctness can be programmatically verified. This creates a strong reward signal for training long-horizon planning and coherence, skills that can generalize to other tasks.

Languages like Lean allow mathematical proofs to be automatically verified. This provides a perfect, binary reward signal (correct/incorrect) for a reinforcement learning agent. It transforms the abstract art of mathematics into a well-defined environment, much like a game of Go, that an AI can be trained to master.

Beyond its academic use for formal verification, Lean is a functional programming language used to build substantial software. Its own tooling is written in Lean, and AWS uses it for a half-million-line compiler for AI accelerators, treating its proof capabilities as a valuable bonus.

Unlike natural language proofs that require human verification, formal systems like Lean allow for automated, verifiable rewards. This could enable an AI to endlessly extend a mathematical library like Mathlib, exploring a vast tree of logic and potentially discovering novel theories without any human check-ins, similar to how AlphaGo trained itself by playing millions of games.

A team successfully formalized and simplified a complex proof from Fields Medalist Peter Scholze using Lean. The tool's interactive feedback guided them, allowing them to refactor and improve the mathematics akin to how a programmer refactors code, even without a complete intuitive grasp of the subject.

The act of creating a formal proof for a piece of software forces a level of rigor that surpasses even implementing it from scratch. This newfound confidence and clarity allows engineers to pursue aggressive optimizations without the fear of introducing subtle bugs, which they would otherwise avoid due to uncertainty.

Expert mathematicians adopt formal tools like Lean not primarily to catch errors, but to offload tedious, low-level deductions. This automation allows them to operate at a higher level of abstraction and focus their cognitive energy on creative intuition and problem-solving strategy.

A major hurdle for formal methods is the effort required to write proofs. Generative AI is becoming capable of producing proofs in formal languages like Lean, which can then be automatically verified by a machine. This could make verified software development scalable for the first time.

Instead of writing code and then tests, developers could define precise mathematical properties. AI would then synthesize both the program and a formal proof that the program meets those specifications, ensuring correctness by design and flipping the current code-first paradigm.