Generative AI can produce the "miraculous" insights needed for formal proofs, like finding an inductive invariant, which traditionally required a PhD. It achieves this by training on vast libraries of existing mathematical proofs and generalizing their underlying patterns, effectively automating the creative leap needed for verification.

Generating truly novel and valid scientific hypotheses requires a specialized, multi-stage AI process. This involves using a reasoning model for idea generation, a literature-grounded model for validation, and a third system for checking originality against existing research. This layered approach overcomes the limitations of a single, general-purpose LLM.

Andrej Karpathy's 'Software 2.0' framework posits that AI automates tasks that are easily *verifiable*. This explains the 'jagged frontier' of AI progress: fields like math and code, where correctness is verifiable, advance rapidly. In contrast, creative and strategic tasks, where success is subjective and hard to verify, lag significantly behind.

AI can produce scientific claims and codebases thousands of times faster than humans. However, the meticulous work of validating these outputs remains a human task. This growing gap between generation and verification could create a backlog of unproven ideas, slowing true scientific advancement.

Unlike traditional software development that starts with unit tests for quality assurance, AI product development often begins with 'vibe testing.' Developers test a broad hypothesis to see if the model's output *feels* right, prioritizing creative exploration over rigid, predefined test cases at the outset.

To ensure scientific validity and mitigate the risk of AI hallucinations, a hybrid approach is most effective. By combining AI's pattern-matching capabilities with traditional physics-based simulation methods, researchers can create a feedback loop where one system validates the other, increasing confidence in the final results.

The tendency for AI models to "make things up," often criticized as hallucination, is functionally the same as creativity. This trait makes computers valuable partners for the first time in domains like art, brainstorming, and entertainment, which were previously inaccessible to hyper-literal machines.

The ultimate goal isn't just modeling specific systems (like protein folding), but automating the entire scientific method. This involves AI generating hypotheses, choosing experiments, analyzing results, and updating a 'world model' of a domain, creating a continuous loop of discovery.

Current LLMs fail at science because they lack the ability to iterate. True scientific inquiry is a loop: form a hypothesis, conduct an experiment, analyze the result (even if incorrect), and refine. AI needs this same iterative capability with the real world to make genuine discoveries.