Technical coding skill ('how to program') is a commodity that can be assisted by LLMs. The real value comes from 'what to program': defining the right clinical question, selecting appropriate data, and designing validation steps. This strategic layer requires deep domain expertise and cannot be fully automated.

Advanced AI models are ineffective in clinical settings without a robust data layer. Ambience had to solve fundamental problems like pulling messy context from inconsistent EHRs and preserving 'decision traces,' which are often destroyed by existing systems with mutable data structures.

AI finds the most efficient correlation in data, even if it's logically flawed. One system learned to associate rulers in medical images with cancer, not the lesion itself, because doctors often measure suspicious spots. This highlights the profound risk of deploying opaque AI systems in critical fields.

Despite the hype, Datycs' CEO finds that even fine-tuned healthcare LLMs struggle with the real-world complexity and messiness of clinical notes. This reality check highlights the ongoing need for specialized NLP and domain-specific tools to achieve accuracy in healthcare.

When a lab report screenshot included a dismissive note about "hemolysis," both human doctors and a vision-enabled AI made the same mistake of ignoring a critical data point. This highlights how AI can inherit human biases embedded in data presentation, underscoring the need to test models with varied information formats.

The concept of a 'correct' clinical output is ambiguous. It requires resolving contradictory chart data, capturing a physician's unstated decision-making, and navigating areas like billing codes where two human experts often disagree. This is a reasoning problem, not just a data problem.

A key risk for AI in healthcare is its tendency to present information with unwarranted certainty, like an "overconfident intern who doesn't know what they don't know." To be safe, these systems must display "calibrated uncertainty," show their sources, and have clear accountability frameworks for when they are inevitably wrong.

Early AI drug discovery platforms built robust models but often failed to generate relevant outputs. Their lack of deep biological understanding led to flawed data collection and training sets, creating a "garbage in, garbage out" problem where models were disconnected from real-world biology.

The primary obstacle preventing healthcare from using its data is not technology but the scarcity of professionals possessing deep expertise in both medicine and data science. This talent gap is the root cause of issues like data silos and complexity, as effectively working with the data requires understanding both domains.