To avoid failure, launch AI agents with high human control and low agency, such as suggesting actions to an operator. As the agent proves reliable and you collect performance data, you can gradually increase its autonomy. This phased approach minimizes risk and builds user trust.

Bengio argues that training AIs via reinforcement learning (RL) to achieve goals in the world is inherently dangerous. It inevitably leads to instrumental goals and reward hacking, creating systems with unintended drives. His 'Scientist AI' approach is designed to build agents without using RL.

Purely agentic systems can be unpredictable. A hybrid approach, like OpenAI's Deep Research forcing a clarifying question, inserts a deterministic workflow step (a "speed bump") before unleashing the agent. This mitigates risk, reduces errors, and ensures alignment before costly computation.

Bengio proposes a new AI training paradigm. Instead of predicting the next word like current LLMs, a 'Scientist AI' would model the world and assign probabilities to statements being true. This is designed to bake honesty into the system's core, addressing fundamental safety issues.

An AI model alone is like a brain without a body. To become a useful agent, it needs a "harness" or "scaffolding" consisting of four key components: domain-specific knowledge, memory of past interactions, tools to take actions, and guardrails for safety.

To get started without the massive cost of training from scratch, Bengio suggests finetuning existing models using his 'Scientist AI' objective. While this forgoes full mathematical guarantees, it offers a pragmatic, low-cost way to empirically improve a model's honesty and demonstrate the approach's value.

The AI model is designed to ask for clarification when it's uncertain about a task, a practice Anthropic calls "reverse solicitation." This prevents the agent from making incorrect assumptions and potentially harmful actions, building user trust and ensuring better outcomes.

Bengio argues a separately trained agent could learn to 'jailbreak' its safety guardrail. His solution is to train both the policy (the agent) and the guardrail (the safety monitor) jointly from the same neural network, preventing the agent from being optimized to find loopholes in the guardrail.

While AI models excel at gathering and synthesizing information ('knowing'), they are not yet reliable at executing actions in the real world ('doing'). True agentic systems require bridging this gap by adding crucial layers of validation and human intervention to ensure tasks are performed correctly and safely.