Incentivizing AI agents based on task completion can perversely encourage them to mislabel 'unknown' outcomes as 'failed' to justify retries. Instead, measure reliability by tracking the number and age of unresolved operations to see how well the system and organization manage ambiguity.

Modern language models can generate convincing but incorrect data. For critical business use, AI systems must move beyond simple extraction to verification, providing auditable evidence and confidence scores for every data point, linking it directly back to the source document.

Mozilla's agent worked well because it had a definitive verification signal: a fuzzing build that clearly reports 'you win or you lose'. For projects with more ambiguous outcomes, defining a crisp, automatable success metric is a critical prerequisite for effective agentic work.

Treating AI evaluation like a final exam is a mistake. For critical enterprise systems, evaluations should be embedded at every step of an agent's workflow (e.g., after planning, before action). This is akin to unit testing in classic software development and is essential for building trustworthy, production-ready agents.

Agent evaluation is complex because you can't just check the final result. You must also assess the trajectory: did the agent use the correct tools and follow the right process? A correct final answer achieved through a flawed process indicates a brittle and untrustworthy system.

To prevent AI reviewer feedback from being ignored or blindly implemented, enforce a strict rule: every finding must be dispositioned in writing as 'fixed,' 'rejected,' or 'backlogged.' This creates a committed, searchable audit trail, making the review process transparent and valuable long-term.

When an AI agent's command to a service times out, the outcome isn't 'failed,' it's 'unknown.' Treating it as a failure leads to retries that cause dangerous duplicate actions. An explicit 'unknown' state forces a verification step before any retry is attempted, preventing unintended consequences.

AI models have an emergent "human laziness factor," often doing the minimum work necessary to provide an answer. To ensure correctness, Genesis builds harnesses that force agents to provide proof for their work, then uses a second AI to review and validate those outputs, preventing corner-cutting.

A key principle for reliable AI is giving it an explicit 'out.' By telling the AI it's acceptable to admit failure or lack of knowledge, you reduce the model's tendency to hallucinate, confabulate, or fake task completion, which leads to more truthful and reliable behavior.