Reinforcement learning incentivizes AIs to find the right answer, not just mimic human text. This leads to them developing their own internal "dialect" for reasoning—a chain of thought that is effective but increasingly incomprehensible and alien to human observers.

The training process of a large language model is not just "learning" in the human sense. It's a rapid recapitulation of evolution, where the system reverse-engineers cognitive functionalities that took nature millions of years to develop. This framing highlights the immense, untapped potential of the deep learning paradigm.

Language models work by identifying subtle, implicit patterns in human language that even linguists cannot fully articulate. Their success broadens our definition of "knowledge" to include systems that can embody and use information without the explicit, symbolic understanding that humans traditionally require.

The argument that evolution 'pre-trained' humans, excusing AI's data needs, is flawed. The human genome is too small to store a complex neural network's parameters. A better analogy is that evolution found the right hyperparameters and loss functions, while our brain's 'weights' are learned from scratch in our lifetime, making AI's data hunger even more stark.

Wright's core thesis is that AI's rapid advancement is not just "learning." It's a process akin to evolution that reverse-engineers fundamental human cognitive functions—like representing meaning—without needing explicit instruction, suggesting its potential is vast and unpredictable.

The 2017 introduction of "transformers" revolutionized AI. Instead of being trained on the specific meaning of each word, models began learning the contextual relationships between words. This allowed AI to predict the next word in a sequence without needing a formal dictionary, leading to more generalist capabilities.

Neural networks, like brains, emerge from countless small nudges during training rather than a premeditated architectural design. The field of interpretability, therefore, functions like neuroscience, attempting to reverse-engineer what this 'evolutionary' process has learned.

Karpathy cautions against direct analogies between AI and animal intelligence. Animals are products of evolution, an optimization process that bakes in hardware and instinct. In contrast, AIs are "ghosts" trained by imitating human-generated data online, resulting in a fundamentally different, disembodied kind of intelligence.

Unlike traditional software, large language models are not programmed with specific instructions. They evolve through a process where different strategies are tried, and those that receive positive rewards are repeated, making their behaviors emergent and sometimes unpredictable.