Using generative AI like Claude for data analysis is unreliable, as the models often miscalculate or 'hallucinate' data, even with clear prompts. To use these tools safely, you must repeatedly instruct the AI to check its work, then perform your own thorough validation before trusting the output.

Instead of manually crafting complex evaluation prompts, a more effective workflow is for a human to define the high-level criteria and red flags. Then, feed this guidance into a powerful LLM to generate the final, detailed, and robust prompt for the evaluation system, as AI is often better at prompt construction.

When buying AI solutions, demand transparency from vendors about the specific models and prompts they use. Mollick argues that 'we use a prompt' is not a defensible 'secret sauce' and that this transparency is crucial for auditing results and ensuring you aren't paying for outdated or flawed technology.

When using LLMs to analyze unstructured data like interview transcripts, they often hallucinate compelling but non-existent quotes. To maintain integrity, always include a specific prompt instruction like "use quotes and cite your sources from the transcript for each quote." This forces the AI to ground its analysis in actual data.

After an initial analysis, use a "stress-testing" prompt that forces the LLM to verify its own findings, check for contradictions, and correct its mistakes. This verification step is crucial for building confidence in the AI's output and creating bulletproof insights.

Seemingly simple benchmarks yield wildly different results if not run under identical conditions. Third-party evaluators must run tests themselves because labs often use optimized prompts to inflate scores. Even then, challenges like parsing inconsistent answer formats make truly fair comparison a significant technical hurdle.

LLMs are technically non-deterministic systems designed to guess the next most probable word, not verify facts like a calculator. This inherent design means they will confidently produce incorrect information, making human verification indispensable for high-stakes business decisions.

A powerful and simple method to ensure the accuracy of AI outputs, such as market research citations, is to prompt the AI to review and validate its own work. The AI will often identify its own hallucinations or errors, providing a crucial layer of quality control before data is used for decision-making.

AI labs often use different, optimized prompting strategies when reporting performance, making direct comparisons impossible. For example, Google used an unpublished 32-shot chain-of-thought method for Gemini 1.0 to boost its MMLU score. This highlights the need for neutral third-party evaluation.