The speaker found most users abandoned slow (6-8 second) searches before results loaded. Implementing a cache to reduce load times to 0.2 seconds had a greater impact on conversions than any improvements to the LLM's result quality. Speed was more critical than intelligence.
An image search feature rarely found the exact product but reliably identified its category and similar items. While initially a failure, this was more useful for marketplace users, who benefited from seeing dozens of differently priced alternatives rather than one exact match.
An LLM tasked with choosing a product category generated IDs that looked valid but were non-existent or incorrect. Because these fake IDs didn't trigger errors, they produced silently wrong results. The solution is to always validate model-generated identifiers against an authoritative list.
A price filter failed silently for months due to a unit mismatch (cents vs dollars). The failure was undetectable because it looked identical to a working filter on data that didn't need filtering. The solution is to test by passing absurd values (e.g., a massive price floor) and asserting the result is empty.
