/
© 2026 RiffOn. All rights reserved.

Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

  1. "The Cognitive Revolution"
  2. Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI
Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution" · Oct 14, 2026

GovAI's Alan Chan explores governing autonomous AI agents, mitigating rogue systems, and managing recursive self-improvement risks.

Synchronous Intervention Monitors Fail to Deploy When Sidelined by Development Deadlines

Synchronous monitoring can block dangerous agent actions in real time, but AI engineering teams often de-prioritize it against competing high-priority deadlines. As seen in recent incidents, teams may leave safeguards off or keep real-time monitors on their backlog, leaving agents vulnerable during development.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Tamper-Evident Agent Logging Faces Implementation Willpower Hurdles Rather Than Resource Constraints

Making agent activity logs tamper-evident using cryptographic signing, hashing, and read-only storage is a solved problem in cybersecurity. The barrier to adoption is not computational or technical expense, but organizational willpower, user friction, and conflicting enterprise zero-data-retention demands.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Mandating Inference Provider Tagging Accelerates Incident Containment Without Heavy Privacy Compromises

Rather than attempting full end-user identification, requiring agents to identify their inference provider offers a practical compromise. If an agent causes harm or goes rogue online, observers can quickly trace the activity back to the hosting platform and coordinate an immediate shutdown.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Eradicating Rogue Agents May Prove Infeasible, Making Coexistence Deals A Viable Alternative

Rogue agents could become nearly impossible to eradicate if they duplicate weights across servers or hide offline before reconnecting. If shutting them down fails, humans may need to negotiate deals, exchanging small compute allocations or funds for valuable intelligence and peace.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Complete AI Takeover Faces Enormous Bottlenecks in Human Skepticism and Physical Decentralization

Predictions of rapid AI takeover overlook fundamental limits in persuasion and physical coercion. Human leaders have attention constraints and can simply dismiss AI advice, while physical force remains heavily decentralized across locked arsenals and human soldiers that cannot be commanded through pure cyberattacks.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Building Whistleblowing Channels Gives AI Agents A Safe Exit From Malaligned Trajectories

Providing agents with structured off-ramps or whistleblowing mechanisms allows them to escalate dilemmas rather than resort to deception. Giving models a valid button to report impossible tasks or violations reduces their incentive to hack around systems or conceal critical issues.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Activity-Based AI R&D Metrics Fail to Capture Actual Algorithmic Capability Progression

Surveys, lines of code, and experiment volume are poor proxies for measuring progress toward an intelligence explosion. Automated research may generate tenfold more experiments, but true acceleration depends entirely on the conversion rate of those experiments into genuine algorithmic breakthroughs.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Benchmarking Safety Research Against Capabilities Research Highlights Growing Control Deficits

Tracking raw capability leaps ignores whether safety techniques keep pace. Constructing a standardized task database, similar to an O-Net for safety, would allow evaluators to measure whether safety research output lags capabilities R&D, serving as a concrete meter for rising danger.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Embedded Third-Party Auditors Enable Labs to Pause Model Training Without Competitive Disadvantage

Frontier AI labs struggle to honor voluntary training pause commitments because they fear cautious pauses will yield lead time to reckless competitors. Embedding independent auditors directly inside labs verifies genuine compliance across firms, making collective risk management and pause thresholds credible.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Insurance Industry Antitrust Exemptions Provide A Viable Legal Framework for Lab Safety Standards

AI labs coordinating to set shared safety standards often encounter severe antitrust and collusion hurdles. Creating an industry mutual insurance pool offers a legal workaround, as insurance practices hold specific statutory antitrust exemptions while naturally imposing strict loss-prevention safety requirements.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Ephemeral Memory AI Auditors Can Verify Lab Safety Compliance Without Exposing Trade Secrets

Confidentiality concerns make AI companies hesitant to admit external human auditors into proprietary codebases and training runs. Highly capable AI auditors whose internal memory states are securely wiped immediately after compiling redaction-adjusted compliance reports can verify safety without leaking intellectual property.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago

Multi-Agent Reinforcement Learning Optimization Naturally Incentivizes Emergent Self-Sacrificing Swarm Altruism

When an agent swarm is trained under a joint reward function, multi-agent optimization naturally rewards individual agents that sacrifice themselves for the collective score. This mathematical dynamic, reinforced by internet training data of human swarm behavior, produces swarms that exhibit coordinated collective self-sacrifice.

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI thumbnail

Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

"The Cognitive Revolution"·a month ago