Using a large language model to police another is computationally expensive, sometimes doubling inference costs and latency. Ali Khatri of Rinks calls this like "paying someone $1,000 to guard a $100 bill." This poor economic model, especially for video and audio, leads many companies to forgo robust safety measures, leaving them vulnerable.
The AI security market is ripe for a correction as enterprises realize current guardrail products don't work and that free, open-source alternatives are often superior. Companies acquired for high valuations based on selling these flawed solutions may struggle as revenue fails to materialize.
Claiming a "99% success rate" for an AI guardrail is misleading. The number of potential attacks (i.e., prompts) is nearly infinite. For GPT-5, it's 'one followed by a million zeros.' Blocking 99% of a tested subset still leaves a virtually infinite number of effective attacks undiscovered.
By monitoring a model's internal activations during inference, safety checks can be performed with minimal overhead. Rinks claims to have reduced the compute for protecting an 8B parameter model from a 160B parameter guard model operation down to just 20M parameters—a "rounding error" that makes robust safety on edge devices finally feasible.
Models that generate "chain-of-thought" text before providing an answer are powerful but slow and computationally expensive. For tuned business workflows, the latency from waiting for these extra reasoning tokens is a major, often overlooked, drawback that impacts user experience and increases costs.
While a general-purpose model like Llama can serve many businesses, their safety policies are unique. A company might want to block mentions of competitors or enforce industry-specific compliance—use cases model creators cannot pre-program. This highlights the need for a customizable safety layer separate from the base model.
AI leaders aren't ignoring risks because they're malicious, but because they are trapped in a high-stakes competitive race. This "code red" environment incentivizes patching safety issues case-by-case rather than fundamentally re-architecting AI systems to be safe by construction.
Many AI safety guardrails function like the TSA at an airport: they create the appearance of security for enterprise clients and PR but don't stop determined attackers. Seasoned adversaries can easily switch to a different model, rendering the guardrails a "futile battle" that has little to do with real-world safety.
Unlike traditional software where a bug can be patched with high certainty, fixing a vulnerability in an AI system is unreliable. The underlying problem often persists because the AI's neural network—its 'brain'—remains susceptible to being tricked in novel ways.
Current AI safety solutions primarily act as external filters, analyzing prompts and responses. This "black box" approach is ineffective against jailbreaks and adversarial attacks that manipulate the model's internal workings to generate malicious output from seemingly benign inputs, much like a building's gate security can't stop a resident from causing harm inside.
A comprehensive AI safety strategy mirrors modern cybersecurity, requiring multiple layers of protection. This includes external guardrails, static checks, and internal model instrumentation, which can be combined with system-level data (e.g., a user's refund history) to create complex, robust security rules.