In China, mayors and governors are promoted based on their ability to meet national priorities. As AI safety becomes a central government goal, these local leaders are now incentivized to create experimental zones and novel regulatory approaches, driving bottom-up policy innovation that can later be adopted nationally.
Related Insights
Top Chinese officials use the metaphor "if the braking system isn't under control, you can't really step on the accelerator with confidence." This reflects a core belief that robust safety measures enable, rather than hinder, the aggressive development and deployment of powerful AI systems, viewing the two as synergistic.
China's binding regulations mean companies focus safety efforts on the 31 specific risks defined by the government. This compliance-driven approach can leave them less prepared for emergent risks like CBRN or loss of control, as resources are directed toward meeting existing legal requirements rather than proactive, voluntary measures.
In China, academics have significant influence on policymaking, partly due to a cultural tradition that highly values scholars. Experts deeply concerned about existential AI risks have briefed the highest levels of government, suggesting that policy may be less susceptible to capture by commercial tech interests compared to the West.
The argument that the U.S. must race to build superintelligence before China is flawed. The Chinese Communist Party's primary goal is control. An uncontrollable AI poses a direct existential threat to their power, making them more likely to heavily regulate or halt its development rather than recklessly pursue it.
The model combines insurance (financial protection), standards (best practices), and audits (verification). Insurers fund robust standards, while enterprises comply to get cheaper insurance. This market mechanism aligns incentives for both rapid AI adoption and robust security, treating them as mutually reinforcing rather than a trade-off.
AI's integration into democracy isn't happening through top-down mandates but via individual actors like city councilors and judges. They can use AI tools for tasks like drafting bills or interpreting laws without seeking permission, leading to rapid, unregulated adoption in areas with low public visibility.
For Chinese policymakers, AI is more than a productivity tool; it represents a crucial opportunity to escape the middle-income trap. They are betting that leadership in AI can fuel the innovation needed to transition from a labor-intensive economy to a developed one, avoiding the stagnation that has plagued other emerging markets.
Unlike the US's voluntary approach, Chinese AI developers must register their models with the government before public release. This involved process requires safety testing against a national standard of 31 risks and giving regulators pre-deployment access for approval, creating a de facto licensing regime for consumer AI.
An FDA-style regulatory model would force AI companies to make a quantitative safety case for their models before deployment. This shifts the burden of proof from regulators to creators, creating powerful financial incentives for labs to invest heavily in safety research, much like pharmaceutical companies invest in clinical trials.
The approach to AI safety isn't new; it mirrors historical solutions for managing technological risk. Just as Benjamin Franklin's 18th-century fire insurance company created building codes and inspections to reduce fires, a modern AI insurance market can drive the creation and adoption of safety standards and audits for AI agents.