The speaker found most users abandoned slow (6-8 second) searches before results loaded. Implementing a cache to reduce load times to 0.2 seconds had a greater impact on conversions than any improvements to the LLM's result quality. Speed was more critical than intelligence.
An LLM tasked with choosing a product category generated IDs that looked valid but were non-existent or incorrect. Because these fake IDs didn't trigger errors, they produced silently wrong results. The solution is to always validate model-generated identifiers against an authoritative list.
An image search feature rarely found the exact product but reliably identified its category and similar items. While initially a failure, this was more useful for marketplace users, who benefited from seeing dozens of differently priced alternatives rather than one exact match.
A price filter failed silently for months due to a unit mismatch (cents vs dollars). The failure was undetectable because it looked identical to a working filter on data that didn't need filtering. The solution is to test by passing absurd values (e.g., a massive price floor) and asserting the result is empty.