When building at the frontier of AI, it's a valid strategy to ship imperfect, "vibe-coded" features. This approach assumes that rapid, near-future model improvements will clean up imperfections, making it better to launch an imperfect product now rather than wait for perfect model performance that is just around the corner.

Technical metrics like "accuracy" are often the wrong measure for ML projects and can mismanage expectations. To achieve success, projects must be evaluated using business KPIs like profit, savings, or ROI. This aligns data science with business goals and reveals the true value of imperfect predictions.

Don't wait for AI to be perfect. The correct strategy is to apply current AI models—which are roughly 60-80% accurate—to business processes where that level of performance is sufficient for a human to then review and bring to 100%. Chasing perfection in-house is a waste of resources given the pace of model improvement.

The benchmark for AI performance shouldn't be perfection, but the existing human alternative. In many contexts, like medical reporting or driving, imperfect AI can still be vastly superior to error-prone humans. The choice is often between a flawed AI and an even more flawed human system, or no system at all.

Users mistakenly evaluate AI tools based on the quality of the first output. However, since 90% of the work is iterative, the superior tool is the one that handles a high volume of refinement prompts most effectively, not the one with the best initial result.

People mistakenly dismiss AI's current inaccuracies as proof of its limitations. This is like calling a stumbling toddler stupid. AI is in a rapid learning phase and will soon be sprinting, creating opportunities for those who understand this developmental stage.

The critical challenge in AI development isn't just improving a model's raw accuracy but building a system that reliably learns from its mistakes. The gap between an 85% accurate prototype and a 99% production-ready system is bridged by an infrastructure that systematically captures and recycles errors into high-quality training data.

Don't aim for a 100% accurate evaluation system. A good system reveals a 'healthy percentage' of incorrect outputs. Getting excited when evals are wrong is key, as each failure is a clear, actionable opportunity to improve your AI agent.

To determine an AI tool's value, ask if you can describe the objective criteria its creators use to improve it. Tools with fast, measurable feedback loops (e.g., code generation passing unit tests) are worth piloting. Those with subjective goals (e.g., writing better fiction) are likely "slop."