As AI tools automate coding and prototyping, the product manager's core function is no longer detailed specification writing. Instead, their value multiplies in judging, facilitating, and making the right strategic decisions quickly. The emphasis moves from the 'how' of building to the 'what' and 'why,' making decision-making the critical skill.

While evals involve testing, their purpose isn't just to report bugs (information), like traditional QA. For an AI PM, evals are a core tool to actively shape and improve the product's behavior and performance (transformation) by iteratively refining prompts, models, and orchestration layers.

Building non-deterministic AI products fundamentally changes the PM role. Instead of creating detailed, rigid specifications, the PM's primary task becomes defining and codifying "what good looks like." This is done by repeatedly grading AI outputs to train evaluation systems and guide the model's behavior.

AI automates tactical tasks, shifting the PM's role from process management to de-risking delivery by developing deep customer insights. This allows PMs to spend more time confirming their instincts about customer needs, which engineering teams now demand.

To make AI adoption tangible, Zapier built rubrics defining "AI fluency" for different roles and seniority levels. By making these skills a measurable part of performance reviews and rewards, you create clear incentives for employees to invest their time in developing them, as behavior follows what gets measured.

AI's rapid capability growth makes top-down product specs obsolete. Product Managers now work bottoms-up with engineers, prototyping and even checking in code using AI tools. This blurs traditional roles, shifting the PM's focus to defining high-level customer needs and evaluating outcomes rather than prescribing features.

Traditional product sense questions are being replaced. AI PM candidates should expect to solve problems live using AI tools or design complex AI-native systems. This shift assesses a candidate's hands-on "builder" capabilities and deep understanding of modern AI architecture.

The rise of AI tools isn't replacing the PM role, but transforming it. PMs who embrace an "AI-enhanced" workflow for research, docs, and prototyping will gain a massive productivity advantage, ultimately displacing those who stick to traditional methods.

AI prototyping should be viewed as a fundamental skill for modern PMs, not an extra job responsibility. Just like using Figma to communicate design, AI prototyping tools allow PMs to make abstract AI concepts tangible for stakeholders and customers. This accelerates feedback loops and improves alignment on complex product behaviors.