Instead of creating rigid systems, formalizing policies makes rules transparent and debatable. It allows for building explicit exceptions, where the final "axiom" in a logical system can simply be "go talk to a human." This preserves necessary flexibility and discretion while making the process auditable and clear.
Related Insights
Use a two-axis framework to determine if a human-in-the-loop is needed. If the AI is highly competent and the task is low-stakes (e.g., internal competitor tracking), full autonomy is fine. For high-stakes tasks (e.g., customer emails), human review is essential, even if the AI is good.
When creating AI governance, differentiate based on risk. High-risk actions, like uploading sensitive company data into a public model, require rigid, enforceable "policies." Lower-risk, judgment-based areas, like when to disclose AI use in an email, are better suited for flexible "guidelines" that allow for autonomy.
Instead of waiting for AI models to be perfect, design your application from the start to allow for human correction. This pragmatic approach acknowledges AI's inherent uncertainty and allows you to deliver value sooner by leveraging human oversight to handle edge cases.
To reliably translate a natural language policy into formal logic, Amazon's system generates multiple translations using an LLM. It then employs a theorem prover to verify these translations are logically equivalent. Mismatches trigger a clarification loop with the user, ensuring the final specification is correct before checking an agent's work.
To ensure reliability in healthcare, ZocDoc doesn't give LLMs free rein. It wraps them in a hybrid system where traditional, deterministic code orchestrates the AI's tasks, sets firm boundaries, and knows when to hand off to a human, preventing the 'praying for the best' approach common with direct LLM use.
Run HR, finance, and legal using AI agents that operate based on codified rules. This creates an autonomous back office where human intervention is only required for exceptions, not routine patterns. The mantra is: "patterns deserve code, exceptions deserve people."
The most effective use of AI isn't full automation, but "hybrid intelligence." This framework ensures humans always remain central to the decision-making process, with AI serving in a complementary, supporting role to augment human intuition and strategy.
For enterprises, scaling AI content without built-in governance is reckless. Rather than manual policing, guardrails like brand rules, compliance checks, and audit trails must be integrated from the start. The principle is "AI drafts, people approve," ensuring speed without sacrificing safety.
Counterintuitively, Uber's AI customer service systems produced better results when given general guidance like "treat your customers well" instead of a rigid, rules-based framework. This suggests that for complex, human-centric tasks, empowering models with common-sense objectives is more effective than micromanagement.
Treat accountability as an engineering problem. Implement a system that logs every significant AI action, decision path, and triggering input. This creates an auditable, attributable record, ensuring that in the event of an incident, the 'why' can be traced without ambiguity, much like a flight recorder after a crash.
