Despite the hype, AI-moderated user interviews are not yet a reliable tool. Even Anthropic, creators of Claude, ran a study with their own AI moderation tool that produced unimpressive, low-quality questions, highlighting the immaturity of the technology.

Human feedback is a 'mirror' reflecting what customers say. Synthetic AI panels are a 'lens' for analyzing existing data to uncover deeper insights without adding to customer survey fatigue. This reframes AI's role from a simple replacement for human access to a new mode of analysis.

Just as one human interview can go off-track, a single AI-generated interview can produce anomalous results. Running a larger batch of synthetic interviews allows you to identify outliers and focus on the "center of gravity" of the responses, increasing the reliability of the overall findings.

Synthetic users, like a stranger at a bar, can provide unfiltered, emotionally rich feedback during simulated interviews. This happens because there's no social barrier or fear of judgment, leading to the discovery of edge cases and deeper motivations that real users might not share with a human interviewer.

A UK startup has found that LLMs can generate accurate, simulated focus group discussions. By creating diverse digital personas, the AI reproduces the nuanced and often surprising feedback that typically requires expensive and slow in-person research, especially in politics.

Unlike general-purpose LLMs (e.g., ChatGPT, Gemini) that produce homogenous answers, Qualtrics's specialized model, trained on survey data, replicates the variability and irrationality inherent in human opinion. This results in more realistic data distributions, preventing the false consensus that generic AI models often create.

The key to reliable AI-powered user research is not novel prompting, but structuring AI tasks to mirror the methodical steps of a human researcher. This involves sequential analysis, verification, and synthesis, which prevents the AI from jumping to conclusions and hallucinating.

Synthetic data removes limitations imposed by human attention spans. For a Booking.com study, a 30-minute survey with a 75-item question—impossible for human respondents—was used to conduct a novel psychographic segmentation. This allows researchers to explore more variables and territories than traditional methods permit.

Don't ask an AI to immediately find themes in open-ended survey responses. First, instruct it to perform "inductive coding"—creating and applying labels to each response based on the data itself. This structured first step ensures a more rigorous and accurate final analysis.