Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Mockapetris calls current AI the second or third coming, recalling a 1985 boom that faded. He says large language models rely on lots of brute force and trial-and-error at a scale people cannot match; a big enough machine can get results without deeper cleverness.

Related Insights

Contrary to the hype, AI isn't a substitute for human thought. It's a powerful pattern-matching tool that consumes vast data. A growing problem is that AI is increasingly training on its own regurgitated output, creating a closed loop that lacks genuine novelty or external grounding.

Today's AI boom is fueled by scaling computation, which is a known engineering challenge. The alternative, embedding nuanced, human-like inductive biases, is far harder as it requires a deep understanding of the problem space. This difficulty gap explains why massive models dominate AI development over more targeted, efficient ones—scaling is simply the more straightforward path.

Current AI, particularly Large Language Models, represents a "brute-force" approach that will soon be obsolete. An expert from semiconductor R&D hub IMEC predicts a software revolution is coming. In a decade, we will look back at today's LLMs with the same amusement we now have for the slow, noisy days of dial-up internet.

The "bitter lesson" in AI research posits that methods leveraging massive computation scale better and ultimately win out over approaches that rely on human-designed domain knowledge or clever shortcuts, favoring scale over ingenuity.

Major AI breakthroughs, like OpenAI's solving of a 200-year-old math problem, are often misunderstood as magical insight. In reality, they represent a massive application of computational leverage—equivalent to tens of thousands of human work-years. AI's value is as an engine for brute-force problem-solving, not a mystical god.

Current language models operate by optimizing statistical patterns without possessing explanatory creativity. Unlike human grandmasters who generate qualitative reasons and novel theories, computers scaling raw calculations cannot produce genuinely new knowledge or solve open-ended problem situations.

True intelligence is adaptive and builds upon partial progress. Terence Tao notes current AIs demonstrate "cleverness" by using trial-and-error at massive scale. They can't yet grab a 'handhold,' stay there, and pull others up—a cumulative process that defines collaborative human intelligence.

Critics dismiss AI for using massive computation over human-like intuition. However, like the Large Hadron Collider, this method uncovers patterns in vast datasets that humans cannot, accelerating breakthroughs in complex fields. The outcome, not the method, is what matters.

Current AI progress isn't true, scalable intelligence but a 'brute force' effort. Amjad Masad contends models improve via massive, manual data labeling and contrived RL environments for specific tasks, a method he calls 'functional AGI,' not a fundamental crack in understanding intelligence.

Unlike traditional software, large language models are not programmed with specific instructions. They evolve through a process where different strategies are tried, and those that receive positive rewards are repeated, making their behaviors emergent and sometimes unpredictable.