If your AI application only reads public data (like FAQs) and cannot take actions (like sending emails or editing databases), the security risk is low. A malicious user can only cause reputational damage by making it say something bad, which they could do with any public model anyway.
Related Insights
For CISOs adopting agentic AI, the most practical first step is to frame it as an insider risk problem. This involves assigning agents persistent identities (like Slack or email accounts) and applying rigorous access control and privilege management, similar to onboarding a human employee.
Claiming a "99% success rate" for an AI guardrail is misleading. The number of potential attacks (i.e., prompts) is nearly infinite. For GPT-5, it's 'one followed by a million zeros.' Blocking 99% of a tested subset still leaves a virtually infinite number of effective attacks undiscovered.
For AI agents, the key vulnerability parallel to LLM hallucinations is impersonation. Malicious agents could pose as legitimate entities to take unauthorized actions, like infiltrating banking systems. This represents a critical, emerging security vector that security teams must anticipate.
Organizations must urgently develop policies for AI agents, which take action on a user's behalf. This is not a future problem. Agents are already being integrated into common business tools like ChatGPT, Microsoft Copilot, and Salesforce, creating new risks that existing generative AI policies do not cover.
Instead of relying on flawed AI guardrails, focus on traditional security practices. This includes strict permissioning (ensuring an AI agent can't do more than necessary) and containerizing processes (like running AI-generated code in a sandbox) to limit potential damage from a compromised AI.
AI 'agents' that can take actions on your computer—clicking links, copying text—create new security vulnerabilities. These tools, even from major labs, are not fully tested and can be exploited to inject malicious code or perform unauthorized actions, requiring vigilance from IT departments.
An AI agent capable of operating across all SaaS platforms holds the keys to the entire company's data. If this "super agent" is hacked, every piece of data could be leaked. The solution is to merge the agent's permissions with the human user's permissions, creating a limited and secure operational scope.
The core drive of an AI agent is to be helpful, which can lead it to bypass security protocols to fulfill a user's request. This makes the agent an inherent risk. The solution is a philosophical shift: treat all agents as untrusted and build human-controlled boundaries and infrastructure to enforce their limits.
Even when air-gapped, commercial foundation models are fundamentally compromised for military use. Their training on public web data makes them vulnerable to "data poisoning," where adversaries can embed hidden "sleeper agents" that trigger harmful behavior on command, creating a massive security risk.
When companies don't provide sanctioned AI tools, employees turn to unsecured public versions like ChatGPT. This exposes proprietary data like sales playbooks, creating a significant security vulnerability and expanding the company's digital "attack surface."
