AI analysis tools tend to focus on the general topic of an interview, often overlooking tangential, unexpected "spiky" details. These anomalies, which pique a human researcher's curiosity, are frequently the source of the most significant product opportunities and breakthroughs.
AI excels at clerical tasks like transcription and basic analysis. However, it lacks the business context to identify strategically important, "spiky" insights. Treat it like a new intern: give it defined tasks, but don't ask it to define your roadmap. It has no practical life experience.
Don't treat evals as a mere checklist. Instead, use them as a creative tool to discover opportunities. A well-designed eval can reveal that a product is underperforming for a specific user segment, pointing directly to areas for high-impact improvement that a simple "vibe check" would miss.
The most valuable consumer insights are not in analytics dashboards, but in the raw, qualitative feedback within social media comments. Winning brands invest in teams whose sole job is to read and interpret this chatter, providing a competitive advantage that quantitative data alone cannot deliver.
AI tools can handle administrative and analytical tasks for product managers, like summarizing notes or drafting stories. However, they lack the essential human elements of empathy, nuanced judgment, and creativity required to truly understand user problems and make difficult trade-off decisions.
While AI efficiently transcribes user interviews, true customer insight comes from ethnographic research—observing users in their natural environment. What people say is often different from their actual behavior. Don't let AI tools create a false sense of understanding that replaces direct observation.
The most effective way to use AI in product discovery is not to delegate tasks to it like an "answer machine." Instead, treat it as a "thought partner." Use prompts that explicitly ask it to challenge your assumptions, turning it into a tool for critical thinking rather than a simple content generator.
When asked to describe a user process, an LLM provides the textbook version. It misses the real-world chaos—forgotten tasks, interruptions, and workarounds. These messy details, which only emerge from talking to real people, are where the most valuable product opportunities are found.
Users often develop multi-product workarounds for issues they don't even recognize as solvable problems. Identifying these subconscious behaviors reveals significant innovation opportunities that users themselves cannot articulate.
Treat AI as a critique partner. After synthesizing research, explain your takeaways and then ask the AI to analyze the same raw data to report on patterns, themes, or conclusions you didn't mention. This is a powerful method for revealing analytical blind spots.
AI can generate hundreds of statistically novel ideas in seconds, but they lack context and feasibility. The bottleneck isn't a lack of ideas, but a lack of *good* ideas. Humans excel at filtering this volume through the lens of experience and strategic value, steering raw output toward a genuinely useful solution.