Tyler Cowen argues the AI risk community's reluctance to engage in formal peer review weakens their arguments. Unlike fields like climate change, which built a robust literature, the movement's reliance on online discourse lacks the rigorous scrutiny needed to build credible scientific consensus.

The FDA receives raw and cleaned datasets from sponsors, not just summary reports. Their internal teams conduct independent analyses, which can lead to findings or data presentations in the official drug label that differ from or expand upon what's in the published paper.

The most valuable lessons in clinical trial design come from understanding what went wrong. By analyzing the protocols of failed studies, researchers can identify hidden biases, flawed methodologies, and uncontrolled variables, learning precisely what to avoid in their own work.

To combat confirmation bias, withhold the final results of an experiment or analysis until the entire team agrees the methodology is sound. This prevents people from subconsciously accepting expected outcomes while overly scrutinizing unexpected ones, leading to more objective conclusions.

Every research paper presented at major conferences is paired with an official critic, or "discussant." This person's job is to translate the work for a broader audience, identify key takeaways, and provide constructive, public feedback, ensuring rigor and clarity.

The danger of LLMs in research extends beyond simple hallucinations. Because they reference scientific literature—up to 50% of which may be irreproducible in life sciences—they can confidently present and build upon flawed or falsified data, creating a false sense of validity and amplifying the reproducibility crisis.

The public appetite for surprising, "Freakonomics-style" insights creates a powerful incentive for researchers to generate headline-grabbing findings. This pressure can lead to data manipulation and shoddy science, contributing to the replication crisis in social sciences as researchers chase fame and book deals.

While commercial conflicts of interest are heavily scrutinized, the pressure on academics to produce positive results to secure their next large institutional grant is often overlooked. This intense pressure to publish favorably creates a significant, less-acknowledged form of research bias.

The CREST trial's positive primary endpoint, assessed by investigators in an open-label setting, was rendered negative upon review by a blinded independent committee. This highlights the critical risk of confirmation bias and the immense weight regulators place on blinded data to determine a drug's true efficacy, especially when endpoints are subjective.