Unlike traditional software where UX can be pre-assessed, AI products are inherently unpredictable. The CEO of Braintrust argues that this makes observability critical. Companies must monitor real-world user interactions to capture failures and successes, creating a data flywheel for rapid improvement.

AI product quality is highly dependent on infrastructure reliability, which is less stable than traditional cloud services. Jared Palmer's team at Vercel monitored key metrics like 'error-free sessions' in near real-time. This intense, data-driven approach is crucial for building a reliable agentic product, as inference providers frequently drop requests.

People overestimate AI's 'out-of-the-box' capability. Successful AI products require extensive work on data pipelines, context tuning, and continuous model training based on output. It's not a plug-and-play solution that magically produces correct responses.

Unlike traditional software, AI products are evolving systems. The role of an AI PM shifts from defining fixed specifications to managing uncertainty, bias, and trust. The focus is on creating feedback loops for continuous improvement and establishing guardrails for model behavior post-launch.

Unlike deterministic SaaS software that works consistently, AI is probabilistic and doesn't work perfectly out of the box. Achieving 'human-grade' performance (e.g., 99.9% reliability) requires continuous tuning and expert guidance, countering the hype that AI is an immediate, hands-off solution.

While 'chain of thought' provides some transparency, advanced inference techniques like speculative decoding are making AI systems less observable. These methods operate on abstract 'hidden states' rather than human-readable text, creating a new challenge for monitoring and debugging that requires specialized tooling.

A core pillar of modern cybersecurity, anomaly detection, fails when applied to AI agents. These systems lack a stable behavioral baseline, making it nearly impossible to distinguish between a harmless emergent behavior and a genuine threat. This requires entirely new detection paradigms.

The durable investment opportunities in agentic AI tooling fall into three categories that will persist across model generations. These are: 1) connecting agents to data for better context, 2) orchestrating and coordinating parallel agents, and 3) providing observability and monitoring to debug inevitable failures.

Companies struggle with AI adoption not because of technology, but because of a lack of trust in probabilistic systems. Platforms like Jetstream are emerging to solve this by creating "AI blueprints"—an operational contract that defines what an AI workflow is supposed to do and flags any deviation, providing necessary control and observability.