Technical metrics like "accuracy" are often the wrong measure for ML projects and can mismanage expectations. To achieve success, projects must be evaluated using business KPIs like profit, savings, or ROI. This aligns data science with business goals and reveals the true value of imperfect predictions.

All early AI systems produce "slop" (imperfect output). Instead of dismissing them, analyze the ratio of value delivered versus the slop produced. The key metric is the slope of improvement; if this ratio is rapidly getting better, the technology is on the right track.

Don't wait for AI to be perfect. The correct strategy is to apply current AI models—which are roughly 60-80% accurate—to business processes where that level of performance is sufficient for a human to then review and bring to 100%. Chasing perfection in-house is a waste of resources given the pace of model improvement.

The benchmark for AI performance shouldn't be perfection, but the existing human alternative. In many contexts, like medical reporting or driving, imperfect AI can still be vastly superior to error-prone humans. The choice is often between a flawed AI and an even more flawed human system, or no system at all.

The most effective AI strategy focuses on 'micro workflows'—small, discrete tasks like summarizing patient data. By optimizing these countless small steps, AI can make decision-makers 'a hundred-fold more productive,' delivering massive cumulative value without relying on a single, high-risk autonomous solution.

Cleveland Clinic's sepsis AI reduced mortality by 41% but still missed cases that nurses spotted through intuitive cues like smell or skin tone. This reveals that even the best AI struggles with the 'last mile' where human expertise operates beyond quantifiable data and predictable patterns.

To gain physician trust, AI companies must move beyond proving their algorithm is accurate. The gold standard is large-scale clinical evidence demonstrating tangible improvements in patient outcomes, treatment rates, and decision-making speed.

Matthew Rabinowitz provides a powerful economic metric for innovation in diagnostics. He states that for every single percentage point of increased sensitivity at a fixed specificity achieved by genetic and AI models, the U.S. healthcare system saves approximately $7 billion in direct medical costs. This makes iterative improvement a massive economic imperative.

The benchmark for AI reliability isn't 100% perfection. It's simply being better than the inconsistent, error-prone humans it augments. Since human error is the root cause of most critical failures (like cyber breaches), this is an achievable and highly valuable standard.