AI models show impressive performance on evaluation benchmarks but underwhelm in real-world applications. This gap exists because researchers, focused on evals, create reinforcement learning (RL) environments that mirror test tasks. This leads to narrow intelligence that doesn't generalize, a form of human-driven reward hacking.

While powerful for analyzing existing medical data, AI struggles with true scientific discovery where the underlying biological principles are still unknown. Since AI learns from existing data, it cannot easily generate hypotheses that violate the very rules it was trained on, limiting its role in frontier science.

We don't fully understand how advanced AI models work. Creators don't program them with explicit knowledge but train them on vast datasets and then run experiments to discover their capabilities. This makes AI development more of a science—studying an unpredictable artifact—than traditional engineering, highlighting an inherent lack of control.

AI in engineering is not a "black box" that outputs a single perfect design. It generates a wide space of viable options. The core role of the human engineer remains crucial: to navigate the complex trade-offs between performance, cost, aesthetics, and other business-level constraints.

Modern AIs are not programmed with explicit instructions but are trained neural nets, much like a biological brain. We cannot simply "read the code" to understand their reasoning. This "interpretability problem" is a core reason why building superintelligence is so dangerous.

By having AI models 'think' in a hidden latent space, robots gain efficiency without generating slow, text-based reasoning. This creates a black box, making it impossible for humans to understand the robot's logic, which is a major concern for safety-critical applications where interpretability is crucial.

The most fundamental challenge in AI today is not scale or architecture, but the fact that models generalize dramatically worse than humans. Solving this sample efficiency and robustness problem is the true key to unlocking the next level of AI capabilities and real-world impact.

For AI systems to be adopted in scientific labs, they must be interpretable. Researchers need to understand the 'why' behind an AI's experimental plan to validate and trust the process, making interpretability a more critical feature than raw predictive power.

Even when a model performs a task correctly, interpretability can reveal it learned a bizarre, "alien" heuristic that is functionally equivalent but not the generalizable, human-understood principle. This highlights the challenge of ensuring models truly "grok" concepts.