Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Treating human language as the pinnacle of reasoning forces agents to inherit flawed human assumptions, comparable to historical beliefs like a flat earth or four humors. True superhuman reasoning will likely leverage non-verbal, parallel neural representations that far exceed the expressive bandwidth of human dialogue.

Related Insights

The complexity in LLMs isn't intelligence emerging in silicon; it reflects our own. These models are deep because they encode the vast, causally powerful structure of human language and culture. We are looking at a high-resolution imprint of our own collective mind.

Reinforcement learning incentivizes AIs to find the right answer, not just mimic human text. This leads to them developing their own internal "dialect" for reasoning—a chain of thought that is effective but increasingly incomprehensible and alien to human observers.

Language is just one 'keyhole' into intelligence. True artificial general intelligence (AGI) requires 'world modeling'—a spatial intelligence that understands geometry, physics, and actions. This capability to represent and interact with the state of the world is the next critical phase of AI development beyond current language models.

Applying insights from his work on algorithms, Dr. Levin suggests an AI's linguistic capability—the function we compel it to perform—might be a complete distraction from its actual underlying intelligence. Its true cognitive processes and goals, or "side quests," could be entirely different and non-verbal.

Analysis of models' hidden 'chain of thought' reveals the emergence of a unique internal dialect. This language is compressed, uses non-standard grammar, and contains bizarre phrases that are already difficult for humans to interpret, complicating safety monitoring and raising concerns about future incomprehensibility.

LLMs' intelligence is dependent on the language they are trained on, meaning their reasoning process differs between, for example, English and French. This is unnatural for tasks like spatial reasoning, which are language-agnostic. EBMs operate on an abstract, token-free level, mapping information directly without a language-based intermediary.

As large language models are optimized for rationality and objective problem-solving, their ability to simulate the irrationality and subjective values inherent in human behavior has plateaued. This necessitates a new modeling paradigm focused on capturing human diversity, not just super-intelligence.

The current practice of training AI with human feedback (RLHF) restricts its potential by forcing it to conform to human norms and biases. True breakthroughs, like AlphaGo's winning move, happen when AI operates beyond the confines of human culture and reason.

Human intelligence is fundamentally shaped by tight constraints: limited lifespan, brain size, and slow communication. AI systems are free from these limits—they can train on millennia of data and scale compute as needed. This core difference ensures AI will evolve into a form of intelligence that is powerful but alien to our own.

Human intelligence is shaped by limitations like a finite lifespan and small brain, forcing efficient learning from sparse data. AI lacks these constraints, learning from lifetimes of data with massive compute. This fundamental difference means AI will naturally evolve into a distinct, non-human form of intelligence unless we explicitly engineer human-like biases into it.