Reflecting on his PhD, Terry Rosen emphasizes that experiments that fail are often the most telling. Instead of discarding negative results, scientists should analyze them deeply. Understanding *why* something didn't work provides critical insights that are essential for iteration and eventual success.

Related Insights

True scientific progress comes from being proven wrong. When an experiment falsifies a prediction, it definitively rules out a potential model of reality, thereby advancing knowledge. This mindset encourages researchers to embrace incorrect hypotheses as learning opportunities rather than failures, getting them closer to understanding the world.

Progress in drug development often hides inside failures. A therapy that fails in one clinical trial can provide critical scientific learnings. One company leveraged insights from a failed study to redesign a subsequent trial, which was successful and led to the drug's approval.

The most valuable lessons in clinical trial design come from understanding what went wrong. By analyzing the protocols of failed studies, researchers can identify hidden biases, flawed methodologies, and uncontrolled variables, learning precisely what to avoid in their own work.

Advice from successful people is inherently flawed because it ignores the role of luck and timing. A more accurate approach is to study failures—the metaphorical planes that didn't return. Understanding why most people *don't* succeed provides a more robust framework for navigating risk than simply copying a survivor's path.

Foster a culture of experimentation by reframing failure. A test where the hypothesis is disproven is just as valuable as a 'win' because it provides crucial user insights. The program's success should be measured by the quantity of quality tests run, not the percentage of successful hypotheses.

Disagreeing with Peter Thiel, Josh Wolf argues that studying people who made willful mistakes is more valuable than studying success stories. Analyzing failures provides a clear catalog of what to avoid, offering a more practical and robust learning framework based on inversion.

Treat your goal as a hypothesis and your actions as inputs. If you don't get the desired outcome, you haven't failed; you've just gathered data showing those inputs were wrong. This shifts the focus from emotional failure to analytical problem-solving about what to change next.

Current LLMs fail at science because they lack the ability to iterate. True scientific inquiry is a loop: form a hypothesis, conduct an experiment, analyze the result (even if incorrect), and refine. AI needs this same iterative capability with the real world to make genuine discoveries.

To truly learn from go-to-market experiments, you can't be half-hearted. StackAI's philosophy is to dedicate significant, focused effort for 1-3 months on a single idea. This ensures that if it fails, you know it's the idea, not poor execution, providing a definitive learning.

The speakers highlight that negative trials in kidney cancer, which showed no benefit to immunotherapy re-challenge, were "super helpful." This is because they provided definitive evidence to stop a common clinical practice that was not helping patients and potentially causing harm, underscoring the constructive role of well-designed "failed" studies.

Failed Scientific Experiments Often Provide the Most Valuable Lessons | RiffOn