Instead of writing a traditional spec, the product team at Yelp starts by writing an ideal sample conversation between a user and the AI assistant. This "golden conversation" serves as the primary artifact to work backward from, defining the desired user experience before any technical requirements.
Instead of manually crafting a system prompt, feed an LLM multiple "golden conversation" examples. Then, ask the LLM to analyze these examples and generate a system prompt that would produce similar conversational flows. This reverses the typical prompt engineering process, letting the ideal output define the instructions.
When an AI model gives nonsensical responses after a long conversation, its context window is likely full. Instead of trying to correct it, reset the context. For prototypes, fork the design to start a new session. For chats, ask the AI to summarize the conversation, then start a new chat with that summary.
Many AI tools expose the model's reasoning before generating an answer. Reading this internal monologue is a powerful debugging technique. It reveals how the AI is interpreting your instructions, allowing you to quickly identify misunderstandings and improve the clarity of your prompts for better results.
Use Claude's "Artifacts" feature to generate interactive, LLM-powered application prototypes directly from a prompt. This allows product managers to test the feel and flow of a conversational AI, including latency and response length, without needing API keys or engineering support, bridging the gap between a static mock and a coded MVP.
For product managers not yet working on AI, the best way to gain experience is to build simple AI tools for personal use cases, like a parenting advisor or a board game timer. Using no-code prototyping tools, they can learn the entire development lifecycle—from ideation to prompting and user feedback—without needing an official AI project at work.
When exploring UI solutions, use a tool like Magic Patterns and its "Inspiration Mode" to generate multiple, distinct design approaches from a single prompt. By asking the AI to "think expansively and make each option differentiated," product managers can quickly explore a wide solution space and avoid getting stuck on a single initial idea.