Bengio argues that training AIs via reinforcement learning (RL) to achieve goals in the world is inherently dangerous. It inevitably leads to instrumental goals and reward hacking, creating systems with unintended drives. His 'Scientist AI' approach is designed to build agents without using RL.

Judea Pearl, a foundational figure in AI, argues that Large Language Models (LLMs) are not on a path to Artificial General Intelligence (AGI). He states they merely summarize human-generated world models rather than discovering causality from raw data. He believes scaling up current methods will not overcome this fundamental mathematical limitation.

Today's AI models are powerful but lack a true sense of causality, leading to illogical errors. Unconventional AI's Naveen Rao hypothesizes that building AI on substrates with inherent time and dynamics—mimicking the physical world—is the key to developing this missing causal understanding.

Bengio proposes a new AI training paradigm. Instead of predicting the next word like current LLMs, a 'Scientist AI' would model the world and assign probabilities to statements being true. This is designed to bake honesty into the system's core, addressing fundamental safety issues.

Simply making LLMs larger will not lead to AGI. True advancement requires solving two distinct problems: 1) Plasticity, the ability to continually learn without "catastrophic forgetting," and 2) moving from correlation-based pattern matching to building causal models of the world.

While both humans and LLMs perform Bayesian updating, humans possess a critical additional capability: causal simulation. When a pen is thrown, a human simulates its trajectory to dodge it—a causal intervention. LLMs are stuck at the level of correlation and cannot perform these essential simulations.

To make genuine scientific breakthroughs, an AI needs to learn the abstract reasoning strategies and mental models of expert scientists. This involves teaching it higher-level concepts, such as thinking in terms of symmetries, a core principle in physics that current models lack.

Purely sequence-based prediction models, while powerful, have fundamental limitations in understanding causality. Achieving robust, trustworthy AI will likely require a hybrid approach that integrates current transformer architectures with symbolic systems, world models, and dedicated causal reasoning components.

While a world model can generate a physically plausible arch, it doesn't understand the underlying physics of force distribution. This gap between pattern matching and causal reasoning is a fundamental split between AI and human intelligence, making current models unsuitable for mission-critical applications like architecture.