AI models are designed to give a complete-sounding answer quickly. To get to a truly great answer, you must challenge their output. Ask "Are you sure this is the best way?" or "What am I not seeing?" to force the AI to perform a deeper, second-level analysis.

An AI that confidently provides wrong answers erodes user trust more than one that admits uncertainty. Designing for "humility" by showing confidence indicators, citing sources, or even refusing to answer is a superior strategy for building long-term user confidence and managing hallucinations.

Users mistakenly evaluate AI tools based on the quality of the first output. However, since 90% of the work is iterative, the superior tool is the one that handles a high volume of refinement prompts most effectively, not the one with the best initial result.

When an AI's response is questionable, go beyond simple re-prompting. Use meta-prompts that explicitly instruct the model to increase its reasoning effort, such as "Think hard about why this is right" or asking for its sources. This can uncover new insights and improve output quality.

Building a functional AI agent is just the starting point. The real work lies in developing a set of evaluations ("evals") to test if the agent consistently behaves as expected. Without quantifying failures and successes against a standard, you're just guessing, not iteratively improving the agent's performance.

Don't aim for a 100% accurate evaluation system. A good system reveals a 'healthy percentage' of incorrect outputs. Getting excited when evals are wrong is key, as each failure is a clear, actionable opportunity to improve your AI agent.

Don't blindly trust AI. The correct mental model is to view it as a super-smart intern fresh out of school. It has vast knowledge but no real-world experience, so its work requires constant verification, code reviews, and a human-in-the-loop process to catch errors.

Comparing AI models based on single, identical prompts is a flawed methodology. A true evaluation involves 'driving' the model through multiple iterations of feedback and correction. This reveals its ability to understand and adapt to your specific intent, which is a far more critical measure of its utility than a single probabilistic output.

To get better results from AI, don't ask for the final output immediately. Instead, prompt the AI to first provide a detailed process. This allows you to review and debug its logic, then instruct it to execute each step for a more accurate outcome.