Go's search space is larger than the number of atoms in the universe, making exhaustive search impossible. AlphaGo's core breakthrough was using neural networks to intelligently guide its search, evaluating only the most promising moves and making an intractable problem solvable.

AlphaGo's architecture mimicked human cognition by pairing a 'fast thinking' neural network for intuition with a 'slow thinking' search algorithm for explicit planning. This hybrid model, combining pattern recognition with calculation, proved more powerful for tackling complex problems than either approach alone.

Modern LLMs use a simple form of reinforcement learning that directly rewards successful outcomes. This contrasts with more sophisticated methods, like those in AlphaGo or the brain, which use "value functions" to estimate long-term consequences. It's a mystery why the simpler approach is so effective.

Instead of training on the single best action from its search (a one-hot label), AlphaGo's policy network learns to imitate the entire probability distribution of moves from MCTS. This 'soft label' contains far more information, enabling a much more effective and sample-efficient form of knowledge distillation.

Monte Carlo Tree Search (MCTS) acts as a 'policy improvement operator.' After the search finds a better move distribution, the policy network is trained to directly predict this improved distribution. This distills the expensive search process into the network itself, making it stronger over time.

Google DeepMind CEO Demis Hassabis argues that today's large models are insufficient for AGI. He believes progress requires reintroducing algorithmic techniques from systems like AlphaGo, specifically planning and search, to enable more robust reasoning and problem-solving capabilities beyond simple pattern matching.

A key insight from AlphaGo is that a relatively shallow neural network can approximate the result of an incredibly deep and complex search tree. This suggests neural nets can learn to compress sequential, recursive computation into a single, efficient forward pass.

Unlike typical reinforcement learning which learns from sparse win/loss signals, AlphaGo's method is remarkably stable. It uses MCTS to generate an 'improved' move for every state, turning the problem into a simple supervised learning task of imitating a better version of itself, avoiding high-variance gradients.

In the endgame, AlphaGo made moves that seemed suboptimal, even giving up points. This was because it wasn't optimizing for a large victory margin (a human heuristic) but purely for maximizing the probability of winning, even by a half-point. This reveals how literal AI objective functions can differ from human proxies for success.