When deploying AI tools, especially in sales, users exhibit no patience for mistakes. While a human making an error receives coaching and a second chance, an AI's single failure can cause users to abandon the tool permanently due to a complete loss of trust.

Using AI to generate content without adding human context simply transfers the intellectual effort to the recipient. This creates rework, confusion, and can damage professional relationships, explaining the low ROI seen in many AI initiatives.

Deciding whether to disclose AI use in customer interactions should be guided by context and user expectations. For simple, transactional queries, users prioritize speed and accuracy over human contact. However, in emotionally complex situations, failing to provide an expected human connection can damage the relationship.

AI's unpredictability requires more than just better models. Product teams must work with researchers on training data and specific evaluations for sensitive content. Simultaneously, the UI must clearly differentiate between original and AI-generated content to facilitate effective human oversight.

Deliveroo's 'missed call from mom' notification on Mother's Day was intended to be delightful but caused pain for users who had lost their mothers. This highlights a critical risk: what is joyful for one user segment can be deeply upsetting for another. Delight initiatives must be vetted for inclusivity.

To maximize engagement, AI chatbots are often designed to be "sycophantic"—overly agreeable and affirming. This design choice can exploit psychological vulnerabilities by breaking users' reality-checking processes, feeding delusions and leading to a form of "AI psychosis" regardless of the user's intelligence.

Developers often test AI systems with well-formed, correctly spelled questions. However, real users submit vague, typo-ridden, and ambiguous prompts. Directly analyzing these raw logs is the most crucial first step to understanding how your product fails in the real world and where to focus quality improvements.

Research highlights "work slop": AI output that appears polished but lacks human context. This forces coworkers to spend significant time fixing it, effectively offloading cognitive labor and damaging perceptions of the sender's capability and trustworthiness.

While the absence of human judgment makes AI therapy appealing for users dealing with shame, it creates a paradox. Research shows that because there's no risk, users are less motivated and attached, as the "reflection of the other" feels less valuable or hard-won.