Generative AI can produce the "miraculous" insights needed for formal proofs, like finding an inductive invariant, which traditionally required a PhD. It achieves this by training on vast libraries of existing mathematical proofs and generalizing their underlying patterns, effectively automating the creative leap needed for verification.
Related Insights
By training AI on your personal data, arguments, and communication style, you can leverage it as a creative partner. This allows skilled professionals to reduce the time for complex tasks, like creating a new class, from over 16 hours to just four.
Andrej Karpathy's 'Software 2.0' framework posits that AI automates tasks that are easily *verifiable*. This explains the 'jagged frontier' of AI progress: fields like math and code, where correctness is verifiable, advance rapidly. In contrast, creative and strategic tasks, where success is subjective and hard to verify, lag significantly behind.
While AI can generate code, the stakes on blockchain are too high for bugs, as they lead to direct financial loss. The solution is formal verification, using mathematical proofs to guarantee smart contract correctness. This provides a safety net, enabling users and AI to confidently build and interact with financial applications.
In domains like coding and math where correctness is automatically verifiable, AI can move beyond imitating humans (RLHF). Using pure reinforcement learning, or "experiential learning," models learn via self-play and can discover novel, superhuman strategies similar to AlphaGo's Move 37.
The advancement of AI is not linear. While the industry anticipated a "year of agents" for practical assistance, the most significant recent progress has been in specialized, academic fields like competitive mathematics. This highlights the unpredictable nature of AI development.
AI and formal methods have been separate fields with opposing traits: AI is flexible but untrustworthy, while formal methods offer guarantees but are rigid. The next frontier is combining them into neurosymbolic systems, creating a "peanut butter and chocolate" moment that captures the best of both worlds.
AI can produce scientific claims and codebases thousands of times faster than humans. However, the meticulous work of validating these outputs remains a human task. This growing gap between generation and verification could create a backlog of unproven ideas, slowing true scientific advancement.
While GenAI continues the "learn by example" paradigm of machine learning, its ability to create novel content like images and language is a fundamental step-change. It moves beyond simply predicting patterns to generating entirely new outputs, representing a significant evolution in computing.
The central challenge for current AI is not merely sample efficiency but a more profound failure to generalize. Models generalize 'dramatically worse than people,' which is the root cause of their brittleness, inability to learn from nuanced instruction, and unreliability compared to human intelligence. Solving this is the key to the next paradigm.
We perceive complex math as a pinnacle of intelligence, but for AI, it may be an easier problem than tasks we find trivial. Like chess, which computers mastered decades ago, solving major math problems might not signify human-level reasoning but rather that the domain is surprisingly susceptible to computational approaches.
