In an AI-driven environment, interfaces are often conversational and nondeterministic rather than static step-by-step UI flows. Francois Lopitaux argues that evaluation systems (evals) become the new PRD. Because product managers cannot dictate the exact UI paths users take, evals are essential to define and validate expected behavior, score accuracy and consistency, and establish clear guardrails for what outputs are acceptable versus unacceptable.
Analytics require deterministic consistency where the same question always returns the exact same answer, but LLMs are inherently creative and probabilistic. To prevent hallucinations and poor analysis, Lopitaux explains that companies must avoid using LLMs indiscriminately and instead build a robust semantic layer or ontology. Grounding the LLM in explicit business definitions counterbalances its probabilistic nature and builds essential user trust.
Because AI allows anyone to rapidly prototype and generate code, it introduces chaotic proliferation. Lopitaux notes that organizations must implement stricter discipline rather than total autonomy. Without strong architectural guardrails, gatekeeping on what enters production, and active pruning of legacy features, rapid code generation will build an unmanageable product that risks killing the company.
When shipping embedded or white-labeled capabilities, customer organizations distribute your features directly to their own end customers and absorb the brand reputation risk. Because of this liability, partner companies do not want surprise capabilities forced on them. Product teams must build granular feature flags and opt-in rollout controls to match the adoption pace of varied enterprise customers.
Unlike traditional software where launching a button marks its completion, probabilistic AI features require ongoing tuning, eval checks, and customer observation. Rather than reorganizing headcount per feature or assigning maintenance overhead after release, ThoughtSpot organizes teams into dedicated product 'tracks.' This ensures that staying with an evolving product, monitoring drifted outputs, and speaking with customers remains a continuous, natural lifecycle.
As agents and AI automate code creation and review, the bottleneck of software shifts entirely to determining what is worth building and validating customer value. Because execution speed is dramatically higher, Lopitaux predicts that development team sizes will shrink, the developer-to-PM ratio will decrease, and the traditional 'two-pizza team' standard will disappear in favor of smaller, agile units.
Because AI tools allow anyone to code, Lopitaux filters associate product manager candidates by asking for their GitHub repository and an explanatory video. This demonstrates their technical fluency, hands-on ability to build, communication and pitching skills, and deep grasp of customer pain points rather than relying on abstract resume credentials.