Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

David Silver argues that human language and trillion-token human datasets act as weights holding back superintelligence. Just as AlphaZero surpassed AlphaGo by abandoning human play data, AI must learn from autonomous experience to invent fields beyond human comprehension.

Related Insights

OpenAI co-founder Ilya Sutskever suggests the path to AGI is not creating a pre-trained, all-knowing model, but an AI that can learn any task as effectively as a human. This reframes the challenge from knowledge transfer to creating a universal learning algorithm, impacting how such systems would be deployed.

Reinforcement learning incentivizes AIs to find the right answer, not just mimic human text. This leads to them developing their own internal "dialect" for reasoning—a chain of thought that is effective but increasingly incomprehensible and alien to human observers.

The boom from LLMs was a 'shortcut' that mined intelligence from existing human data. This has limits. To achieve novel breakthroughs beyond that corpus, the field now re-integrates the original DeepMind philosophy of agents learning through interaction (like reinforcement learning) to generate truly new knowledge.

Language is just one 'keyhole' into intelligence. True artificial general intelligence (AGI) requires 'world modeling'—a spatial intelligence that understands geometry, physics, and actions. This capability to represent and interact with the state of the world is the next critical phase of AI development beyond current language models.

In domains like coding and math where correctness is automatically verifiable, AI can move beyond imitating humans (RLHF). Using pure reinforcement learning, or "experiential learning," models learn via self-play and can discover novel, superhuman strategies similar to AlphaGo's Move 37.

The two greatest AI achievements are generative AI (mimicking human knowledge) and deep reinforcement learning (discovering superhuman strategies). The grand challenge, and the future of AI, is to fuse these two threads into a single system that can both leverage existing knowledge and innovate beyond it.

By removing all human game data and learning only from self-play, AlphaZero first rediscovered human strategies and then discarded them for superior, 'alien' ones. This showed that relying solely on human data can limit an AI's potential, anchoring it to existing knowledge and cognitive biases.

Sutton contends that focusing on supervised learning is a mistake when trying to build AGI. Animals, including humans, do not learn core skills like walking or seeing via labeled examples. True intelligence is rooted in experiential, goal-driven learning, making school-style supervision an irrelevant model for core AI.

Treating human language as the pinnacle of reasoning forces agents to inherit flawed human assumptions, comparable to historical beliefs like a flat earth or four humors. True superhuman reasoning will likely leverage non-verbal, parallel neural representations that far exceed the expressive bandwidth of human dialogue.

The host predicts superintelligence won't just be a better reasoner but will come from merging latent spaces of different data types (text, vision, physics). This will give AI an intuitive, non-verbal "feel" for complex domains, much like a human knows where their arm is without calculation.