According to IBM, the key barrier preventing agentic AI systems from moving from impressive demos to widespread production is not a lack of technical capability. The real challenge is the absence of appropriate governance structures and operating models needed to scale these systems safely and effectively.
A new academic framework, ArbiterK, challenges the standard model of an LLM acting as the central controller. It inverts the paradigm by embedding the LLM within a deterministic execution system, demoting it to a suggestion engine. This ensures the system, not the probabilistic LLM, retains final control and enforces rules.
Relying on prompt engineering for safety is insufficient and easily bypassed. The expert consensus is to build safeguards directly into the system's architecture. Architectural controls are immutable during runtime, whereas prompt-level controls can be manipulated or overridden by clever user inputs.
The conversation around Agentic AI has matured beyond abstract policies. The consensus among consultancies, tech firms, and academics is that effective governance requires embedding controls, like access management and validation, directly into the system's architecture as a core design principle.
For regulated industries like banking, Boston Consulting Group and OpenAI advocate for a centralized middleware layer, or 'control plane.' This architectural component acts as a single gateway through which all AI systems must operate, enabling consistent oversight, standardized controls, and auditable governance across the entire organization.
Frameworks from firms like KPMG and AWS emphasize that AI agents must be treated as entities with identities and permissions. A strong IAM foundation is a critical control layer to prevent agents from accessing or unintentionally leaking sensitive information, reflecting a broader shift to treat agents like any other privileged user in an IT ecosystem.