Generative AI can produce the "miraculous" insights needed for formal proofs, like finding an inductive invariant, which traditionally required a PhD. It achieves this by training on vast libraries of existing mathematical proofs and generalizing their underlying patterns, effectively automating the creative leap needed for verification.

Languages like Lean allow mathematical proofs to be automatically verified. This provides a perfect, binary reward signal (correct/incorrect) for a reinforcement learning agent. It transforms the abstract art of mathematics into a well-defined environment, much like a game of Go, that an AI can be trained to master.

An AI model disproved a mathematical conjecture not through a flash of creative genius, but by methodically applying a known technique from a different math subfield. This highlights AI's current strength: synthesizing vast, disparate human knowledge rather than generating truly novel, alien ideas. It's an exhaustive librarian, not an intuitive genius.

A common fear is that AIs will produce billion-line proofs of theorems without offering human insight. However, an alternative and perhaps more likely future is that their superhuman capabilities will be applied to explanation. They could take complex, human-incomprehensible proofs and find novel ways to make them intuitive and easy to understand.

Expert mathematicians adopt formal tools like Lean not primarily to catch errors, but to offload tedious, low-level deductions. This automation allows them to operate at a higher level of abstraction and focus their cognitive energy on creative intuition and problem-solving strategy.

Top mathematician Timothy Gowers was relieved an AI model disproved a conjecture with a counterexample rather than proving it, considering the former an 'easier' task. This reaction from one of the world's smartest people highlights the palpable and imminent arrival of superhuman intelligence.

Unlike medicine or biology, which require messy, expensive real-world experiments, pure mathematics offers a cost-effective and prestigious arena for AI labs to demonstrate their models' abstract reasoning power. A proof is a proof, requiring no lab work or physical trials to validate.

AI models tend to produce short, clever mathematical proofs. This is likely not a sign of elegance, but a limitation. They lack the ability to reliably verify their own correctness over long, complex arguments, so they are constrained to producing outputs that are short enough to be checked by humans or other systems.

Simply generating a mathematical proof in natural language is useless because it could be thousands of pages long and contain subtle errors. The pivotal innovation was combining AI reasoning with formal verification. This ensures the output is provably correct and usable, solving the critical problems of trust and utility for complex, AI-generated work.