Anthropic's response to its security leak by citing "human error" highlights a coming trend. As AI systems become more autonomous, corporations will find it easier to attribute failures to human oversight rather than the complex, black-box nature of their AI, creating a new liability dynamic.

Customers are hesitant to trust a black-box AI with critical operations. The winning business model is to sell a complete outcome or service, using AI internally for a massive efficiency advantage while keeping humans in the loop for quality and trust.

A crucial function for humans in an AI-driven economy is to serve as a target for lawsuits. Because you can't easily sue a data center, regulated professions will require a 'human in the loop' to take legal responsibility. This creates a valuable economic role for humans: being a legally accountable entity.

When an AI agent errs in a medical or financial context, it is legally unclear who is liable: the AI lab, the deploying company, or the end-user. This novel legal problem, which challenges a century of precedent, creates significant friction and will slow agent adoption in regulated industries.

Pure software-as-a-service (SaaS) companies are vulnerable to being replaced by foundational AI models that can replicate their functionality. A Sequoia partner suggests the defensible model is to become a services company that uses technology as a layer, focusing on implementation, strategy, and human expertise.

As AI agents take over execution, the primary human role will evolve to setting constraints and shouldering the responsibility for agent decisions. Every employee will effectively become a manager of an AI team, with their main function being risk mitigation and accountability, turning everyone into a leader responsible for agent outcomes.

Despite the potential for AI to create more efficient legal services, new tech-first law firms face significant hurdles. The established reputation of a major law firm ("the name on the letterhead") sends a powerful signal in litigation. Furthermore, incumbent firms carry malpractice insurance, meaning they assume liability for mistakes—a crucial function AI startups cannot easily replicate.

While clients may use AI tools like ChatGPT to check agency work, it won't eliminate the need for expert service. Instead, AI raises the bar. Clients will expect more efficiency and better results for their money, and will crave a deeper, consultative human partnership to navigate the new complexity.

When a highly autonomous AI fails, the root cause is often not the technology itself, but the organization's lack of a pre-defined governance framework. High AI independence ruthlessly exposes any ambiguity in responsibility, liability, and oversight that was already present within the company.