A major pitfall for brands is using generative AI to autonomously create large volumes of product descriptions. This low-quality "AI slop" lacks value, erodes brand image, and harms sales performance. AI's better use is in targeted data enrichment and discovery.

Generative AI is designed for creative generation, not consistent output. This core feature makes it unreliable for critical, live applications without human oversight. Humans require predictable patterns, which current AI alone cannot guarantee, making a human at the helm essential for safety and trust.

After several iterations, the fortune-telling app started overusing the word "rock," generating similar fortunes about finding rocks that look like pizza or cupcakes. This highlights how generative AI can fixate on a theme, demonstrating the need for human testing and curation to ensure variety and quality.

AI makes it easy to generate mediocre content, shrinking the gap between bad and passable. However, the effort required to create truly good, differentiating content has increased, widening the gap between what is passable and what is excellent, making true differentiation more difficult.

AI tools rarely produce perfect results initially. The user's critical role is to serve as a creative director, not just an operator. This means iteratively refining prompts, demanding better scripts, and correcting logical flaws in the output to avoid generic, low-quality content.

Many product builders overestimate current AI capabilities. Understanding AI's limitations, like the non-deterministic nature of LLMs, is more critical than knowing its strengths. Overstating AI's capacity is a direct path to product failure and bad investments.

To manage non-deterministic AI products, Shopify created an internal tool where PMs grade AI-generated outputs. This creates a "ground truth" dataset of what "good" looks like, which is then used to fine-tune a separate LLM that acts as an automated quality judge for new features and updates.

AI can generate vast amounts of content, but its value is limited by our ability to verify its accuracy. This is fast for visual outputs (images, UI) where our eyes instantly spot flaws, but slow and difficult for abstract domains like back-end code, math, or financial data, which require deep expertise to validate.

GM's CMO warns that AI in creative often produces average results because it finds the "most likely next answer," reflecting the category norm, not a distinctive brand voice. Simple edits can also trigger a full re-render, introducing new errors and creating more work.