Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Tom says he is sure that rerunning Niagara, Energize, and the pembrolizumab trial would not produce the same two negatives and one positive. He argues randomized phase 3 trials are imperfect and vulnerable to noise, chance, and unseen imbalances. That should temper how confidently we interpret any single result.

Related Insights

The CREST trial showed benefit driven by patients with carcinoma in situ (CIS), while the Potomac trial showed a lack of benefit in the same subgroup. This stark inconsistency demonstrates that subgroup analyses, even for stratified factors, can be unreliable and are a weak basis for regulatory decisions or label restrictions.

The standard of care for melanoma is improving so quickly that control arms in recent clinical trials are significantly outperforming historical data. This 'rising tide' effect makes it increasingly difficult for new drugs to show a statistically significant benefit, creating high unpredictability for developers.

While smaller trials like KEYNOTE-905 can show dramatic results, they are subject to more statistical noise. Larger, thousand-patient studies like B15 and Niagara, with narrower confidence intervals, are considered closer to the true effect size and provide a more stable foundation for establishing the standard of care.

The COSMIC-313 trial is cited as a 'negative positive study' because it met its primary endpoint, but the overall results for response and survival were underwhelming, especially given the toxicity. This highlights a critical trap in drug development: a statistically significant result is not always clinically meaningful.

The most valuable lessons in clinical trial design come from understanding what went wrong. By analyzing the protocols of failed studies, researchers can identify hidden biases, flawed methodologies, and uncontrolled variables, learning precisely what to avoid in their own work.

"Proof" isn't about absolute certainty but about managing error. It requires setting an evidence bar that balances two risks: believing false things and discounting true things. Clinical trials, for example, penalize a false positive four times more than a false negative.

Reflecting on a past failure, Isaac Stoner asserts that the costliest risk in drug development is ambiguity. A poorly designed study yielding a “thumb sideways” is worse than a well-designed one that produces a clear failure, as ambiguity prevents actionable decisions and wastes invaluable time and capital.

The practice-changing KEYNOTE-689 trial was open-label, meaning patients knew their treatment. This could introduce bias; patients on the standard care arm may have dropped out ("bailed"), while those on the pembrolizumab arm might have progressed, artificially making the rates of patients reaching surgery appear similar.

Even when trials like LITESPARK 022 and Keynote 564 use identical eligibility criteria, outdated staging systems result in patient populations with different underlying risks. This makes direct comparison of outcomes between trials, even for the same drug, an unfair and statistically flawed analysis that ignores the function of a control arm.

Experts believe the stark difference in complete response rates (5% vs 30%) between two major ADC trials is likely due to "noise"—variations in patient populations (e.g., more upper tract disease) and stricter central review criteria, rather than a fundamental difference in the therapies' effectiveness.