To find talent capable of managing an AI stack, traditional interviews are insufficient. A better test is to provide candidates with platform credits (e.g., Replit) and challenge them to build a functional agent that automates a real business task, proving their practical skills.

With LLMs making remote coding tests unreliable, the new standard is face-to-face interviews focused on practical problems. Instead of abstract algorithms, candidates are asked to fix failing tests or debug code, assessing their real-world problem-solving skills which are much harder to fake.

To accurately assess candidates, interviews must be split. One part must be a "Zero AI" test to evaluate raw problem-solving ability and foundational knowledge, complete with cheat detection. The other part must be an "AI-Max" test to assess their skill in leveraging AI tools to be a "roboticist."

To assess a product manager's AI skills, integrate AI into your standard hiring process rather than just asking theoretical questions. Expect candidates to use AI tools in take-home case studies and analytical interviews to test for practical application and raise the quality bar.

To build an AI-native team, shift the hiring process from reviewing resumes to evaluating portfolios of work. Ask candidates to demonstrate what they've built with AI, their favorite prompt techniques, and apps they wish they could create. This reveals practical skill over credentialism.

To assess a candidate's ability to use AI as a thinking partner, have them solve a problem with an LLM. The key is observing their follow-up prompts and their ability to guide the AI step-by-step, rather than just accepting the initial output.

To discern a true AI-native product manager from a tourist, ask what they have built or automated. The ability to point to specific agents created or workflows automated demonstrates deep, practical expertise, which is far more valuable than just discussing AI concepts.

Glean has updated its interview process to screen for "AI fluency" across all departments. They don't expect expertise. Instead, they test for curiosity and initiative by asking candidates how they've personally used AI, looking for a mindset that embraces new ways of working.

Since AI assistants make it easy for candidates to complete take-home coding exercises, simply evaluating the final product is no longer an effective screening method. The new best practice is to require candidates to build with AI and then explain their thought process, revealing their true engineering and problem-solving skills.