The attack on the widely used LightLLM package demonstrates a major software supply chain vulnerability. Malicious code inserted into a routine update silently stole credentials from countless AI tools, a risk that will be amplified by autonomous AI agents.

Defenders of AI models are "fighting against infinity" because as model capabilities and complexity grow, the potential attack surface area expands faster than it can be secured. This gives attackers a persistent upper hand in the cat-and-mouse game of AI security.

The same AI technology amplifying cyber threats can also generate highly secure, formally verified code. This presents a historic opportunity for a society-wide effort to replace vulnerable legacy software in critical infrastructure, leading to a durable reduction in cyber risk. The main challenge is creating the motivation for this massive undertaking.

As powerful open-source AI models from China (like Kimi) are adopted globally for coding, a new threat emerges. It's possible to embed secret prompts that inject malicious or corrupted code into software at a massive scale. As AI writes more code, human oversight becomes impossible, creating a significant vulnerability.

The productivity gains from AI incentivize companies to ship work without full verification. While rational for an individual firm, this practice introduces a "Trojan Horse" of subtle flaws and technical debt at a massive scale, creating accumulating systemic risk across the economy.

AI agents can generate and merge code at a rate that far outstrips human review. While this offers unprecedented velocity, it creates a critical challenge: ensuring quality, security, and correctness. Developing trust and automated validation for this new paradigm is the industry's next major hurdle.

A former OpenAI security expert argues that even if AI makes codebases more secure, hacking won't become harder. Attackers exploit the entire system—runtime behavior, configurations, authentication—not just static code. Looking only at code is like seeing a dinosaur's bones; you miss the muscles, feathers, and behavior that define the real-world attack surface.

AI 'agents' that can take actions on your computer—clicking links, copying text—create new security vulnerabilities. These tools, even from major labs, are not fully tested and can be exploited to inject malicious code or perform unauthorized actions, requiring vigilance from IT departments.

Moltbook was reportedly created by an AI agent instructed to build a social network. This "bot vibe coding" resulted in a system with massive, easily exploitable security holes, highlighting the danger of deploying unaudited AI-generated infrastructure.