Instead of writing a traditional spec, the product team at Yelp starts by writing an ideal sample conversation between a user and the AI assistant. This "golden conversation" serves as the primary artifact to work backward from, defining the desired user experience before any technical requirements.

Related Insights

Effective prompt engineering for AI agents isn't an unstructured art. A robust prompt clearly defines the agent's persona ('Role'), gives specific, bracketed commands for external inputs ('Instructions'), and sets boundaries on behavior ('Guardrails'). This structure signals advanced AI literacy to interviewers and collaborators.

Instead of manually crafting a system prompt, feed an LLM multiple "golden conversation" examples. Then, ask the LLM to analyze these examples and generate a system prompt that would produce similar conversational flows. This reverses the typical prompt engineering process, letting the ideal output define the instructions.

To discover high-value AI use cases, reframe the problem. Instead of thinking about features, ask, "If my user had a human assistant for this workflow, what tasks would they delegate?" This simple question uncovers powerful opportunities where agents can perform valuable jobs, shifting focus from technology to user value.

In an age of rapid AI prototyping, it's easy to jump to solutions without deeply understanding the problem. The act of writing a spec forces product managers to clarify their thinking and structure context. Writing is how PMs "refactor their thoughts" and avoid overfitting to a partially-baked solution.

Traditional "writing-first" cultures create communication gaps and translation errors. With modern AI tools, product managers can now build working prototypes in hours. This "show, don't tell" approach gets ideas validated faster, secures project leadership, and overcomes language and team barriers.

When building complex AI systems that mediate human interactions, like an AI proctor, start by creating a service map for the ideal human-to-human experience. Define what a great real-world proctor would do and say, then use that blueprint to design the AI's behavior, ensuring it's grounded in human needs.

The prompts for your "LLM as a judge" evals function as a new form of PRD. They explicitly define the desired behavior, edge cases, and quality standards for your AI agent. Unlike static PRDs, these are living documents, derived from real user data and are constantly, automatically testing if the product meets its requirements.

A powerful but unintuitive AI development pattern is to give a model a vague goal and let it attempt a full implementation. This "throwaway" draft, with its mistakes and unexpected choices, provides crucial insights for writing a much more accurate plan for the final version.

Instead of writing detailed Product Requirement Documents (PRDs), use a brief prompt with an AI tool like Vercel's v0. The generated prototype immediately reveals gaps and unstated assumptions in your thinking, allowing you to refine requirements based on the AI's 'misinterpretations' before creating a clearer final spec.

Instead of immediately building, engage AI in a Socratic dialogue. Set rules like "ask one question at a time" and "probe assumptions." This structured conversation clarifies the problem and user scenarios, essentially replacing initial team brainstorming sessions and creating a better final prompt for prototyping tools.

Yelp PMs Design AI Agents by First Writing "Golden Conversations" | RiffOn