Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Modern AI models are grown through gradient descent tuning across trillions of variables rather than crafted through human-written instructions. Because developers only author the optimization rule rather than the internal logic, they cannot know how those resulting parameters actually produce complex model behaviors.

Related Insights

Frontier AI models start as randomly initialized networks and learn via trial and error. This process creates complex, opaque internal representations that are not directly understood by their creators. This makes the analogy of 'growing' them more accurate than 'engineering' them like traditional, inspectable software.

Advanced AI techniques like 'recurrent depth' make models more efficient but also less transparent. They process information without an easily readable 'chain of thought,' making it harder for researchers to monitor their reasoning. This creates a direct and worrying trade-off between capability and safety.

Unlike traditional software where features are explicitly coded, frontier AI systems are trained on vast datasets, leading to emergent abilities. Their internal mechanisms are not directly designed, which is why developers struggle to reliably instill intended goals and prevent unwanted behaviors.

AI development is more like farming than engineering. Companies create conditions for models to learn but don't directly code their behaviors. This leads to a lack of deep understanding and results in emergent, unpredictable actions that were never explicitly programmed.

Unlike traditional software, AI models are not explicitly programmed line-by-line. They self-organize from massive datasets in a process more akin to growth. This creates a "black box" of billions of incomprehensible numbers, meaning even their developers cannot fully explain or verify their internal reasoning or safety.

We don't fully understand how advanced AI models work. Creators don't program them with explicit knowledge but train them on vast datasets and then run experiments to discover their capabilities. This makes AI development more of a science—studying an unpredictable artifact—than traditional engineering, highlighting an inherent lack of control.

OpenAI's leadership is calling for a slowdown because AI is no longer programmed but "grown." Its capability to self-improve is outpacing our ability to ensure alignment, creating an unpredictable and potentially uncontrollable feedback loop that even its creators don't fully understand.

Modern AIs are not programmed with explicit instructions but are trained neural nets, much like a biological brain. We cannot simply "read the code" to understand their reasoning. This "interpretability problem" is a core reason why building superintelligence is so dangerous.

The core safety challenge is that we have little understanding of how advanced AI systems function internally. We are essentially "growing" them through training, not engineering them with comprehensible parts. This means we cannot verify their true goals, making safety measures a gamble on observed behavior.

Unlike traditional software, large language models are not programmed with specific instructions. They evolve through a process where different strategies are tried, and those that receive positive rewards are repeated, making their behaviors emergent and sometimes unpredictable.