Treating AI evaluation like a final exam is a mistake. For critical enterprise systems, evaluations should be embedded at every step of an agent's workflow (e.g., after planning, before action). This is akin to unit testing in classic software development and is essential for building trustworthy, production-ready agents.

The non-deterministic nature of agentic AI makes traditional pass/fail testing insufficient. Businesses must adopt a multi-dimensional scorecard for every interaction, evaluating metrics like compliance, factual accuracy, latency, and intent recognition, not just task completion.

Building a functional AI agent is just the starting point. The real work lies in developing a set of evaluations ("evals") to test if the agent consistently behaves as expected. Without quantifying failures and successes against a standard, you're just guessing, not iteratively improving the agent's performance.

General-purpose AI assistants produce inconsistent output. Instead, define AI agents with specific roles, boundaries, and quality gates, much like onboarding a new engineer with a clear job description. This disciplined approach leverages how LLMs are trained, leading to more reliable and predictable results within the SDLC.

Don't aim for a 100% accurate evaluation system. A good system reveals a 'healthy percentage' of incorrect outputs. Getting excited when evals are wrong is key, as each failure is a clear, actionable opportunity to improve your AI agent.

A one-size-fits-all evaluation method is inefficient. Use simple code for deterministic checks like word count. Leverage an LLM-as-a-judge for subjective qualities like tone. Reserve costly human evaluation for ambiguous cases flagged by the LLM or for validating new features.

Many people struggle to define what 'good' looks like. Building an evaluation (eval) for an AI system requires you to codify your quality standards, forcing a level of clarity and commitment that improves your own process and the AI's output.

Counterintuitively, Uber's AI customer service systems produced better results when given general guidance like "treat your customers well" instead of a rigid, rules-based framework. This suggests that for complex, human-centric tasks, empowering models with common-sense objectives is more effective than micromanagement.

When an AI agent performs poorly, the most effective solution isn't clever prompt engineering. Braintrust's CEO's strategy is to "close the session" and rewrite the evaluation script from scratch. This forces clarity on the definition of success, which is often the root cause of the agent's failure.