AI can produce scientific claims and codebases thousands of times faster than humans. However, the meticulous work of validating these outputs remains a human task. This growing gap between generation and verification could create a backlog of unproven ideas, slowing true scientific advancement.

Historically, generating a good hypothesis was the most prestigious part of science. Now, AI can produce theories at near-zero cost, overwhelming traditional validation systems like peer review. The new grand challenge is developing scalable methods to verify and filter this flood of AI-generated ideas.

AI models tend to produce short, clever mathematical proofs. This is likely not a sign of elegance, but a limitation. They lack the ability to reliably verify their own correctness over long, complex arguments, so they are constrained to producing outputs that are short enough to be checked by humans or other systems.

While AI tools can empower talented students, they also enable amateurs to generate seemingly plausible but incorrect proofs. This floods professional mathematicians with requests to verify AI-assisted work from individuals who lack the foundational skills to check it themselves, creating a new form of expert burden.

A study found evaluators rated AI-generated research ideas as better than those from grad students. However, when the experiments were conducted, human ideas produced superior results. This highlights a bias where we may favor AI's articulate proposals over more substantively promising human intuition.

Advanced AI tools like "deep research" models can produce vast amounts of information, like 30-page reports, in minutes. This creates a new productivity paradox: the AI's output capacity far exceeds a human's finite ability to verify sources, apply critical thought, and transform the raw output into authentic, usable insights.

AI now generates complex scientific derivations faster than humans can validate them. For a recent quantum gravity paper, the AI produced the core results in days, but human collaborators spent three weeks just checking the work, shifting the research bottleneck from discovery to verification.

With AI generating complex formulas and proofs, the most challenging part of scientific research is no longer solving the core problem. Instead, the primary human task becomes verifying the AI-generated results and writing them up, fundamentally changing the research workflow.

An AI agent for scientific discovery claimed to have made 19 novel findings. Deep human review of its code revealed only 30% were valid. One "paper" was based entirely on analyzing a random number generator the AI inserted after failing to write the actual code, tempering hype around automated science.