The Cleveland Clinic's sepsis model reduced mortality by 41% despite not being perfect. This case study demonstrates that AI's true utility is in achieving substantial, incremental gains over existing systems, rather than the hyped-up promise of flawless, silver-bullet solutions.

All early AI systems produce "slop" (imperfect output). Instead of dismissing them, analyze the ratio of value delivered versus the slop produced. The key metric is the slope of improvement; if this ratio is rapidly getting better, the technology is on the right track.

The benchmark for AI performance shouldn't be perfection, but the existing human alternative. In many contexts, like medical reporting or driving, imperfect AI can still be vastly superior to error-prone humans. The choice is often between a flawed AI and an even more flawed human system, or no system at all.

In a partnership with Kenya's Penda Health, OpenAI conducted the first randomized controlled trial of an LLM co-pilot for physicians. The study demonstrated a statistically significant improvement in diagnosis and treatment outcomes for patients whose doctors used the AI assistant. This provides crucial real-world evidence that AI can move beyond lab benchmarks to tangibly improve care.

The most effective AI strategy focuses on 'micro workflows'—small, discrete tasks like summarizing patient data. By optimizing these countless small steps, AI can make decision-makers 'a hundred-fold more productive,' delivering massive cumulative value without relying on a single, high-risk autonomous solution.

Proving the ROI of clinical AI can take years if based solely on patient outcomes. Instead, focus on early, measurable operational wins that are known proxies for better care. Track metrics like increased clinician capacity and higher patient engagement rates to prove the system's value and build momentum.

Cleveland Clinic's sepsis AI reduced mortality by 41% but still missed cases that nurses spotted through intuitive cues like smell or skin tone. This reveals that even the best AI struggles with the 'last mile' where human expertise operates beyond quantifiable data and predictable patterns.

To gain physician trust, AI companies must move beyond proving their algorithm is accurate. The gold standard is large-scale clinical evidence demonstrating tangible improvements in patient outcomes, treatment rates, and decision-making speed.

Acing a medical exam is a misleading benchmark for an AI's clinical readiness. The crucial metric isn't general knowledge but proven performance on specific, high-risk tasks. Patients need to know how an AI performed in thousands of similar past procedures, not its score on a multiple-choice test.