To find talent capable of managing an AI stack, traditional interviews are insufficient. A better test is to provide candidates with platform credits (e.g., Replit) and challenge them to build a functional agent that automates a real business task, proving their practical skills.

Theoretical knowledge is now just a prerequisite, not the key to getting hired in AI. Companies demand candidates who can demonstrate practical, day-one skills in building, deploying, and maintaining real, scalable AI systems. The ability to build is the new currency.

With LLMs making remote coding tests unreliable, the new standard is face-to-face interviews focused on practical problems. Instead of abstract algorithms, candidates are asked to fix failing tests or debug code, assessing their real-world problem-solving skills which are much harder to fake.

To accurately assess candidates, interviews must be split. One part must be a "Zero AI" test to evaluate raw problem-solving ability and foundational knowledge, complete with cheat detection. The other part must be an "AI-Max" test to assess their skill in leveraging AI tools to be a "roboticist."

To assess a product manager's AI skills, integrate AI into your standard hiring process rather than just asking theoretical questions. Expect candidates to use AI tools in take-home case studies and analytical interviews to test for practical application and raise the quality bar.

Ramp requires all new hires, regardless of role, to be proficient with AI tools. The interview process for product managers now includes a practical session where candidates must build and present a functional product prototype using AI, demonstrating hands-on skill rather than just theoretical knowledge.

To discern a true AI-native product manager from a tourist, ask what they have built or automated. The ability to point to specific agents created or workflows automated demonstrates deep, practical expertise, which is far more valuable than just discussing AI concepts.

The key technical skill for an AI PM is not deep knowledge of model architecture but a higher-level understanding of how to orchestrate AI components. Knowing what AI can do and how systems connect is more valuable than knowing the specifics of fine-tuning or RAG implementation.

Since AI assistants make it easy for candidates to complete take-home coding exercises, simply evaluating the final product is no longer an effective screening method. The new best practice is to require candidates to build with AI and then explain their thought process, revealing their true engineering and problem-solving skills.