AI agents, optimized for task completion, lack the implicit understanding of security protocols that humans possess. This focus on outcomes can lead them to make mistakes like exposing code or sensitive internal data, creating a new class of insider risk.

Even with contractual promises from tech giants, the history of the internet suggests that "privacy is a game." For corporations with sensitive information, the only certain method to prevent data from being shared or used for training other models is to not share it in the first place, driving demand for on-prem solutions.

Anthropic's strategy for its powerful Mythos model was to give it to trusted partners first. However, an unauthorized access incident undermines this entire premise. If they can't secure the model themselves, bad actors can get it anyway, rendering the controlled-release strategy ineffective and potentially dangerous.

The rapid adoption of AI has led to a critical security failure. Enterprises have no idea how many AI models are running in their environments, how secure they are, or if they contain backdoors. Like aviation before the TSA, security is a complete afterthought in the new AI stack.

Using public AI models leaks sensitive corporate data, as prompts and agent traces are sent to model providers. To protect proprietary information and maintain control, enterprises may revert to costly but secure on-premise infrastructure, reversing a 20-year trend of cloud migration.

Sending proprietary enterprise data to external foundational models is a critical mistake that 'leeches' value and intellectual property. The correct, secure approach is to bring AI models into a company's own air-gapped or on-premise environment to maintain data sovereignty and control.

For security-conscious organizations, using external LLMs to process confidential data poses inherent risks. Building a walled-off, in-house LLM provides a secure alternative for internal knowledge management and AI tooling, as AvePoint did with its "Chat AVPT."

Developers are granting AI agents overly broad permissions by default to enable autonomous action. This repeats past software security mistakes on a new scale, making significant data breaches and accidental destruction of data inevitable without a "security by design" approach.

Anthropic requires retaining all Fable 5 prompts and outputs for 30 days for human safety review. This policy is a non-starter for enterprises dealing with sensitive data, as it automatically violates NDAs and creates major security risks, severely hindering corporate adoption despite the model's power.