Despite mandated adoption and new capabilities, there's no clear evidence yet that AI prototyping tools lead to faster production or better software. The time spent building a highly-detailed interactive prototype may not be quicker than traditional methods, and the complexity requires rigorous code review.
Related Insights
Prototyping directly in the production environment makes high-quality interactions achievable without extensive resources. This dissolves the traditional design dilemma of sacrificing quality for speed, allowing teams to build better products faster.
Contrary to claims that "handoff is dead," designers at top companies use AI-generated prototypes as highly detailed specs. These interactive prototypes provide more information than static designs but are still handed off to developers for implementation, rather than being merged directly into production.
There's a significant gap between AI performance on structured benchmarks and its real-world utility. A randomized controlled trial (RCT) found that open-source software developers were actually slowed down by 20% when using AI assistants, despite being miscalibrated to believe the tools were helping. This highlights the limitations of current evaluation methods.
Human intuition is a poor gauge of AI's actual productivity benefits. A study found developers felt significantly sped up by AI coding tools even when objective measurements showed no speed increase. The real value may come from enabling tasks that otherwise wouldn't be attempted, rather than simply accelerating existing workflows.
Simply deploying AI to write code faster doesn't increase end-to-end velocity. It creates a new bottleneck where human engineers are overwhelmed with reviewing a flood of AI-generated code. To truly benefit, companies must also automate verification and validation processes.
AI co-pilots have accelerated engineering velocity to the point where traditional design-led workflows are now the slowest part of product development. In response, some agile teams are flipping the process, having engineers build a functional prototype first and then creating formal Figma designs and UI polish later.
Historically, resource-intensive prototyping (requiring designers and tools like Figma) was reserved for major features. AI tools reduce prototype creation time to minutes, allowing PMs to de-risk even minor features with user testing and solution discovery, improving the entire product's success rate.
While AI coding assistants appear to boost output, they introduce a "rework tax." A Stanford study found AI-generated code leads to significant downstream refactoring. A team might ship 40% more code, but if half of that increase is just fixing last week's AI-generated "slop," the real productivity gain is much lower than headlines suggest.
For creative work like design, AI's true value isn't just accelerating tasks. It's enabling designers to explore a much wider option space, test more possibilities, and apply more craft to the final choice. Since design is non-deterministic, AI serves creative exploration more than simple speed.
The panel suggests a best practice for AI prototyping tools: focus on pinpointed interactions or small, specific user flows. Once a prototype grows to encompass the entire product, it's more efficient to move directly into the codebase, as you're past the point of exploration.
