Because PMs deeply understand the customer's job, needs, and alternatives, they are the only ones qualified to write the evaluation criteria for what a successful AI output looks like. This critical task goes beyond technical metrics and is core to the PM's role in the AI era.
When building its "Underlord" agent, Descript rushed into a private alpha with a deliberately diverse user base, including both novices and experts in AI and video editing. This exposed them to real-world, non-expert language and use cases, preventing them from over-optimizing for their own internal jargon and assumptions.
The journey from individual contributor to VP of Product at Descript wasn't about formal promotions. Instead, it was a gradual process of adding so much value in product discussions that she was invited into progressively more strategic meetings. When you're consistently indispensable in "the room," you eventually belong there permanently.
PMs at founder-led startups often fail to gain influence by jumping straight to strategy. The key is to first earn deep credibility by mastering the product, its customers, and the business. Only after you've demonstrated this command will a founder trust your strategic instincts. Don't skip the tactical work of earning your seat at the table.
To set realistic success metrics for new AI tools, Descript used its most popular pre-AI feature, "remove filler words," as the baseline. They compared adoption and retention of new AI features against this known winner, providing a clear, internal benchmark for what "good" looks like instead of guessing at targets.
Twice in her career, including for her role at Descript that led to her becoming CEO, Laura Burkhouser landed a job by simply finding a product she fell in love with as a user and cold-emailing to ask for a job. Instead of optimizing for title or money, she optimized for learning and passion, which ultimately led to greater success.
Descript's AI audio tool worsened after they trained it on extremely bad audio (e.g., vacuum cleaners). They learned the model that best fixes terrible audio is different from the one that best improves merely "okay" audio—the more common user scenario. You must train for your primary user's reality, not the worst possible edge case.