Before going live, top teams run the AI system in parallel with existing workflows, processing real production traffic without exposing the output. This "shadow mode" provides an honest accuracy benchmark on unfiltered data and is treated as a non-negotiable step to de-risk the launch.

Snowflake's CEO rejects a "YOLO AI" approach where model outputs are unpredictable. He insists enterprise AI products must be trustworthy, treating their development with the same discipline as software engineering. This includes mandatory evaluations (evals) for every model change to ensure reliability.

The speed of AI development has created a paradoxical situation where the time to release a new model is shorter than the time required to conduct comprehensive, long-running tests on the previous version. This necessitates new evaluation frameworks, like a 'recall program' for API-based models.

Teams often mistakenly debate between using offline evals or online production monitoring. This is a false choice. Evals are crucial for testing against known failure modes before deployment. Production monitoring is essential for discovering new, unexpected failure patterns from real user interactions. Both are required for a robust feedback loop.

Treating AI evaluation like a final exam is a mistake. For critical enterprise systems, evaluations should be embedded at every step of an agent's workflow (e.g., after planning, before action). This is akin to unit testing in classic software development and is essential for building trustworthy, production-ready agents.

People overestimate AI's 'out-of-the-box' capability. Successful AI products require extensive work on data pipelines, context tuning, and continuous model training based on output. It's not a plug-and-play solution that magically produces correct responses.

Unlike traditional software, AI products are evolving systems. The role of an AI PM shifts from defining fixed specifications to managing uncertainty, bias, and trust. The focus is on creating feedback loops for continuous improvement and establishing guardrails for model behavior post-launch.

Treating AI risk management as a final step before launch leads to failure and loss of customer trust. Instead, it must be an integrated, continuous process throughout the entire AI development pipeline, from conception to deployment and iteration, to be effective.

Shift the view of AI from a singular product launch to a continuous process encompassing use case selection, training, deployment, and decommissioning. This broader aperture creates multiple intervention points to embed responsibility and mitigate harm throughout the lifecycle.