The 'FDA for AI' analogy is flawed because the FDA's rigid, one-drug-one-disease model is ill-suited for a general-purpose technology. This structure struggles with modern personalized medicine, and a similar top-down regime for AI could embed faulty assumptions, stifling innovation and adaptability for a rapidly evolving field.
Related Insights
The common analogy of AI to electricity is dangerously rosy. AI is more like fire: a transformative tool that, if mismanaged or weaponized, can spread uncontrollably with devastating consequences. This mental model better prepares us for AI's inherent risks and accelerating power.
The emphasis on long-term, unprovable risks like AI superintelligence is a strategic diversion. It shifts regulatory and safety efforts away from addressing tangible, immediate problems like model inaccuracy and security vulnerabilities, effectively resulting in a lack of meaningful oversight today.
Instead of trying to legally define and ban 'superintelligence,' a more practical approach is to prohibit specific, catastrophic outcomes like overthrowing the government. This shifts the burden of proof to AI developers, forcing them to demonstrate their systems cannot cause these predefined harms, sidestepping definitional debates.
The current trend of building huge, generalist AI systems is fundamentally mismatched for specialized applications like mental health. A more tailored, participatory design process is needed instead of assuming the default chatbot interface is the correct answer.
An "AI arms race" is underway where stakeholders apply AI to broken, adversarial processes. The true transformation comes from reinventing these workflows entirely, such as moving to real-time payment adjudication where trust is pre-established, thus eliminating the core conflict that AI is currently used to fight over.
As AI allows any patient to generate well-reasoned, personalized treatment plans, the medical system will face pressure to evolve beyond rigid standards. This will necessitate reforms around liability, data access, and a patient's "right to try" non-standard treatments that are demonstrably well-researched via AI.
The idea of individual states creating their own AI regulations is fundamentally flawed. AI operates across state lines, making it a clear case of interstate commerce that demands a unified federal approach. A 50-state regulatory framework would create chaos and hinder the country's ability to compete globally in AI development.
An FDA-style regulatory model would force AI companies to make a quantitative safety case for their models before deployment. This shifts the burden of proof from regulators to creators, creating powerful financial incentives for labs to invest heavily in safety research, much like pharmaceutical companies invest in clinical trials.
The approach to AI safety isn't new; it mirrors historical solutions for managing technological risk. Just as Benjamin Franklin's 18th-century fire insurance company created building codes and inspections to reduce fires, a modern AI insurance market can drive the creation and adoption of safety standards and audits for AI agents.
The central challenge for current AI is not merely sample efficiency but a more profound failure to generalize. Models generalize 'dramatically worse than people,' which is the root cause of their brittleness, inability to learn from nuanced instruction, and unreliability compared to human intelligence. Solving this is the key to the next paradigm.
