The "low-hanging fruit" argument for diminishing returns in science is flawed because it assumes a static problem space. Progress is often explosive when entirely new fields, like computer science, emerge from other domains, opening up a fresh landscape of easy problems where rapid breakthroughs are once again possible.

The orbital anomaly of Uranus correctly led to the discovery of Neptune, strengthening Newtonian theory. A similar anomaly in Mercury's orbit was only explained by General Relativity. This highlights a core challenge in science: you cannot know beforehand whether an anomaly requires a small fix or a complete paradigm shift.

Kepler's method of testing numerous, often strange, hypotheses against Tycho Brahe's precise data mirrors how AIs can generate and verify countless ideas. This uncovers empirical regularities that can later fuel deeper theoretical understanding, much like Newton's laws explained Kepler's findings.

A new scientific theory isn't valuable if it only recategorizes what we already know. Its true merit lies in suggesting an outrageous, unique, and testable experiment that no other existing theory could conceive of. Without this, it's just a reframing of old ideas.

Current AI can learn to predict complex patterns, like planetary orbits, from data. However, it struggles to abstract the underlying causal laws, such as Newtonian physics (F=MA). This leap to a higher level of abstraction remains a fundamental challenge beyond simple pattern recognition.

Long before Einstein's relativity, scholars like Pierre-Simon Laplace and John Michell theorized about "dark stars." They reasoned that if a star were massive enough, its escape velocity could exceed the speed of light, trapping light and rendering it invisible. This early concept was based entirely on Newton's laws of gravity, demonstrating remarkable scientific foresight.

Copernicus's simpler heliocentric model was less accurate than the highly-tweaked Ptolemaic system. This shows that progress isn't linear accuracy; a new, conceptually superior framework might perform worse at first. It requires further refinement, as Kepler provided for Copernicus, to realize its full potential.

Initially, the Copernican model was neither simpler (it had more epicycles) nor more observationally accurate than the established Ptolemaic system. The scientific community embraced it centuries before definitive proof, highlighting that progress can be driven by a theory's perceived explanatory potential, not just immediate empirical superiority.

Peter Kaufman proposes a 'three bucket' framework to validate ideas. A principle is trustworthy if it consistently appears across the 13.7 billion-year history of the inorganic universe (physics), the 3.5 billion years of biology, and the 20,000 years of recorded human history. This method uses large, relevant sample sizes to confirm universal truths.