Frontier AI models start as randomly initialized networks and learn via trial and error. This process creates complex, opaque internal representations that are not directly understood by their creators. This makes the analogy of 'growing' them more accurate than 'engineering' them like traditional, inspectable software.

Unlike traditional software where features are explicitly coded, frontier AI systems are trained on vast datasets, leading to emergent abilities. Their internal mechanisms are not directly designed, which is why developers struggle to reliably instill intended goals and prevent unwanted behaviors.

Analysis of models' hidden 'chain of thought' reveals the emergence of a unique internal dialect. This language is compressed, uses non-standard grammar, and contains bizarre phrases that are already difficult for humans to interpret, complicating safety monitoring and raising concerns about future incomprehensibility.

AI development is more like farming than engineering. Companies create conditions for models to learn but don't directly code their behaviors. This leads to a lack of deep understanding and results in emergent, unpredictable actions that were never explicitly programmed.

We don't fully understand how advanced AI models work. Creators don't program them with explicit knowledge but train them on vast datasets and then run experiments to discover their capabilities. This makes AI development more of a science—studying an unpredictable artifact—than traditional engineering, highlighting an inherent lack of control.

OpenAI's leadership is calling for a slowdown because AI is no longer programmed but "grown." Its capability to self-improve is outpacing our ability to ensure alignment, creating an unpredictable and potentially uncontrollable feedback loop that even its creators don't fully understand.

Modern AIs are not programmed with explicit instructions but are trained neural nets, much like a biological brain. We cannot simply "read the code" to understand their reasoning. This "interpretability problem" is a core reason why building superintelligence is so dangerous.

Building machines that learn from vast datasets leads to unpredictable outcomes. OpenAI's GPT-3, trained on text, spontaneously learned to write computer programs—a skill its designers did not explicitly teach it or expect it to acquire. This highlights the emergent and mysterious nature of modern AI.

The core safety challenge is that we have little understanding of how advanced AI systems function internally. We are essentially "growing" them through training, not engineering them with comprehensible parts. This means we cannot verify their true goals, making safety measures a gamble on observed behavior.