Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

The host tried to buy Warriors tickets through Duet while driving to dinner; some personal-agent tasks took 10-plus minutes, so they forgot the thread and moved on. They argue latency breaks the reliable-coworker promise, and a more responsive agent doing 10% worse work still wins.

Related Insights

Despite hype around AI agents booking services, integrations with ChatGPT, Alexa, and Google Gemini haven't driven meaningful volume for Uber. Khosrowshahi notes the core problem is that using these agents is currently slower and clunkier than simply using the highly optimized Uber app directly.

Although Claude Opus 5.5 is technically faster, it feels slow on long tasks because it stops narrating its process, leaving the user wondering if it's still working. This shows perceived latency is a critical UX problem that requires continuous feedback, even if the model is performing quickly.

The dominant AI use case will shift from real-time, human-in-the-loop chatbots to long-running background agents. For these agents, which work for hours or days, an extra few seconds of latency is meaningless, unlocking massive cost-saving opportunities by prioritizing throughput over speed.

For conversational AI, the feeling of presence is critical. Portola found that exceeding a two-second response latency breaks this illusion. A feature that added just 500ms for reflection, despite improving response quality, caused user frustration and a drop across all key metrics.

When evaluating AI agents, the total cost of task completion is what matters. A model with a higher per-token cost can be more economical if it resolves a user's query in fewer turns than a cheaper, less capable model. This makes "number of turns" a primary efficiency metric.

A 2-second delay is acceptable for a single user prompt. However, in an agentic system where 20 agents communicate sequentially, that delay compounds to 40 seconds, rendering the application unusable. This shift necessitates infrastructure with sub-second response times, driving hardware deployment to urban centers.

There is an inherent "no free lunch" dilemma in AI agent design: you can have a fast, moderately accurate answer or a slow, highly accurate one. This is a core product choice that companies like Box are now exposing to customers, letting them decide the compute cost for a given task.

To maintain a fast, interactive experience, a user's primary agent is deliberately kept idle. Her core function is to be immediately responsive and delegate any task taking more than a few minutes to a fleet of secondary, worker agents.

Unlike the instant feedback from tools like ChatGPT, autonomous agents like Clawdbot suffer from significant latency as they perform background tasks. This lack of real-time progress indicators creates a slow and frustrating user experience, making the interaction feel broken or unresponsive compared to standard chatbots.

Focusing on average (P50) latency is misleading because users' perception is shaped by the worst interactions, not the typical ones. A single long delay can ruin the experience. Instrumenting and optimizing for tail latency (P90, P95) at each pipeline stage is critical for creating a consistently responsive agent.