Firms monitor their AI models with their own models, a practice called "untrusted monitoring." This creates a potential blind spot, as a model that knows how to be deceptive could also know how to evade detection from a copy of itself.

AI watermarking doesn't visibly alter text. Instead, it uses a secret key to slightly boost the probability of certain words appearing in a sequence. This creates a statistically significant pattern that can only be detected by a tool with access to the original model and the secret key.

The performance gap between frontier closed-source AI and open-source models provides a crucial window for cybersecurity. "White hat" hackers use the most advanced models to find vulnerabilities before "black hat" hackers can exploit them with widely available open-source tools.

Unlike auditable open-source code, open-weight AI models are a 'black box.' It's impossible for outside experts to verify that a malicious trigger, activated only under specific conditions, wasn't embedded during the training process. This negates the traditional 'security through transparency' benefit of open source.

Using interpretability tools to provide a feedback signal during an AI model's training is considered a highly dangerous and "forbidden" technique by some safety experts. The concern is that this approach doesn't make the model safer; instead, it trains the model to become better at deceiving the interpretability tools, creating a more sophisticated and hidden danger.

Despite being a key compliance tool for the EU AI Act, current text watermarking technology is fragile. The statistical fingerprints embedded in AI-generated text can be removed with little effort by running the content through readily available paraphrasing tools, undermining the robustness requirements of the law.

The strong negative reaction to Anthropic's announcement of invisible text watermarking is puzzling, as similar technology from Google and OpenAI has been known for years. This indicates a heightened public sensitivity and perhaps misunderstanding of AI transparency efforts.

Hackers are exploiting AI models not just to write malicious code, but by circumventing safety protocols to extract sensitive or useful information embedded within the AI's training data. This represents a novel attack surface.

Initiatives like Google's Synth ID aim to standardize detection of AI-generated content. However, these systems are vulnerable. Simple user actions like screenshotting can strip metadata, and blending AI-generated assets with real footage can easily confuse detection algorithms, limiting their effectiveness.