Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Jon Krohn says a model can now generate hundreds of plausible proofs or thousands of lines of code in hours, so generation stops being the bottleneck; verification takes over. He argues the results that hold up will pair machine-checkable evidence, like Lean proofs or strong test suites, with humans who understand them.

Related Insights

OpenAI's team found that as code generation speed approaches real-time, the new constraint is the human capacity to verify correctness. The challenge shifts from creating code to reviewing and testing the massive output to ensure it's bug-free and meets requirements.

Publishing unvetted machine proofs creates chaotic races where researchers risk duplicate work trying to decode difficult outputs. Instead of dumping raw solutions, AI companies are increasingly likely to contract professional mathematicians to interpret, translate, and verify arguments before release.

Current AI models struggle to reliably verify their own mathematical steps. To establish trustworthy proofs, researchers now pair large language models with separate formal software like the Lean language, which verifies the logical validity of generated code against strict mathematical axioms.

AI can produce scientific claims and codebases thousands of times faster than humans. However, the meticulous work of validating these outputs remains a human task. This growing gap between generation and verification could create a backlog of unproven ideas, slowing true scientific advancement.

A major hurdle for formal methods is the effort required to write proofs. Generative AI is becoming capable of producing proofs in formal languages like Lean, which can then be automatically verified by a machine. This could make verified software development scalable for the first time.

Historically, generating a good hypothesis was the most prestigious part of science. Now, AI can produce theories at near-zero cost, overwhelming traditional validation systems like peer review. The new grand challenge is developing scalable methods to verify and filter this flood of AI-generated ideas.

AI can generate vast amounts of content, but its value is limited by our ability to verify its accuracy. This is fast for visual outputs (images, UI) where our eyes instantly spot flaws, but slow and difficult for abstract domains like back-end code, math, or financial data, which require deep expertise to validate.

Simply generating a mathematical proof in natural language is useless because it could be thousands of pages long and contain subtle errors. The pivotal innovation was combining AI reasoning with formal verification. This ensures the output is provably correct and usable, solving the critical problems of trust and utility for complex, AI-generated work.

AI now generates complex scientific derivations faster than humans can validate them. For a recent quantum gravity paper, the AI produced the core results in days, but human collaborators spent three weeks just checking the work, shifting the research bottleneck from discovery to verification.

With AI generating complex formulas and proofs, the most challenging part of scientific research is no longer solving the core problem. Instead, the primary human task becomes verifying the AI-generated results and writing them up, fundamentally changing the research workflow.