The same AI technology amplifying cyber threats can also generate highly secure, formally verified code. This presents a historic opportunity for a society-wide effort to replace vulnerable legacy software in critical infrastructure, leading to a durable reduction in cyber risk. The main challenge is creating the motivation for this massive undertaking.

Related Insights

The rapid evolution of AI makes reactive security obsolete. The new approach involves testing models in high-fidelity simulated environments to observe emergent behaviors from the outside. This allows mapping attack surfaces even without fully understanding the model's internal mechanics.

While AI can generate code, the stakes on blockchain are too high for bugs, as they lead to direct financial loss. The solution is formal verification, using mathematical proofs to guarantee smart contract correctness. This provides a safety net, enabling users and AI to confidently build and interact with financial applications.

AI tools aren't just lowering the bar for novice hackers; they are making experts more effective, enabling attacks at a greater scale across all stages of the "cyber kill chain." AI is a universal force multiplier for offense, making even powerful reverse engineers shockingly more effective.

In large enterprises, AI adoption creates a conflict. The CTO pushes for speed and innovation via AI agents, while the CISO worries about security risks from a flood of AI-generated code. Successful devtools must address this duality, providing developer leverage while ensuring security for the CISO.

Vercel is building infrastructure based on a threat model where developers cannot be trusted to handle security correctly. By extracting critical functions like authentication and data access from the application code, the platform can enforce security regardless of the quality or origin (human or AI) of the app's code.

Systems like the legal and tax systems assume human-level effort, making them vulnerable to denial-of-service attacks from AI. An AI can generate millions of lawsuits or tax filings, overwhelming the infrastructure. Society must redesign these foundational systems with the assumption that they will face persistent, large-scale, intelligent attacks.

Generative AI's positive impact on cybersecurity spending stems from three distinct drivers: it massively expands the digital "surface area" needing protection (more code, more agents), it elevates the threat environment by empowering adversaries, and it introduces new data governance and regulatory challenges.

Security's focus shifted from physical (bodyguards) to digital (cybersecurity) with the internet. As AI agents become primary economic actors, security must undergo a similar fundamental reinvention. The core business value may be the same (like Blockbuster vs. Netflix), but the security architecture must be rebuilt from first principles.

The benchmark for AI reliability isn't 100% perfection. It's simply being better than the inconsistent, error-prone humans it augments. Since human error is the root cause of most critical failures (like cyber breaches), this is an achievable and highly valuable standard.

The goal for trustworthy AI isn't simply open-source code, but verifiability. This means having mathematical proof, like attestations from secure enclaves, that the code running on a server exactly matches the public, auditable code, ensuring no hidden manipulation.

AI Will Enable a 'Great Rewrite' of Society's Code to Erase Decades of Vulnerabilities | RiffOn