Insurers lack the historical loss data required to price novel AI risks. The solution is to use red teaming and systematic evaluations to create a large pool of "synthetic data" on how an AI product behaves and fails. This data on failure frequency and severity can be directly plugged into traditional actuarial models.

The model combines insurance (financial protection), standards (best practices), and audits (verification). Insurers fund robust standards, while enterprises comply to get cheaper insurance. This market mechanism aligns incentives for both rapid AI adoption and robust security, treating them as mutually reinforcing rather than a trade-off.

While foundation models carry systemic risk, AI applications make "thicker promises" to enterprises, like guaranteeing specific outcomes in customer support. This specificity creates more immediate and tangible business risks (e.g., brand disasters, financial errors), making the application layer the primary area where trust and insurance are needed now.

Organizations must urgently develop policies for AI agents, which take action on a user's behalf. This is not a future problem. Agents are already being integrated into common business tools like ChatGPT, Microsoft Copilot, and Salesforce, creating new risks that existing generative AI policies do not cover.

Drawing from the nuclear energy insurance model, the private market cannot effectively insure against massive AI tail risks. A better model involves the government capping liability (e.g., above $15B), creating a backstop that allows a private insurance market to flourish and provide crucial governance for more common risks.

Insurers like AIG are seeking to exclude liabilities from AI use, such as deepfake scams or chatbot errors, from standard corporate policies. This forces businesses to either purchase expensive, capped add-ons or assume a significant new category of uninsurable risk.

Security's focus shifted from physical (bodyguards) to digital (cybersecurity) with the internet. As AI agents become primary economic actors, security must undergo a similar fundamental reinvention. The core business value may be the same (like Blockbuster vs. Netflix), but the security architecture must be rebuilt from first principles.

AI and big data give insurers increasingly precise information on individual risk. As they approach perfect prediction, the concept of insurance as risk-pooling breaks down. If an insurer knows your house will burn down and charges an equivalent premium, you're no longer insured; you're just pre-paying for a disaster.

The approach to AI safety isn't new; it mirrors historical solutions for managing technological risk. Just as Benjamin Franklin's 18th-century fire insurance company created building codes and inspections to reduce fires, a modern AI insurance market can drive the creation and adoption of safety standards and audits for AI agents.