When using LLMs to judge other models' output, they consistently rate towards the middle of the curve, akin to humans giving a generic "7 out of 10." These AI judges are not "spiky" enough, failing to recognize unique or exceptional qualities that a human evaluator with strong taste would identify.

Rather than optimizing solely for performance on standard industry benchmarks, Ideogram focuses on embedding a subjective quality of "taste" into its models. This requires using human designers for evaluation, as they believe current AI is poor at judging aesthetic nuances, giving them a unique creative edge.

Creating AI that can reliably judge aesthetics is a frontier problem. Unlike tasks with clear right or wrong answers, aesthetics is subjective. This lack of a clear, objective benchmark makes it difficult to apply standard model improvement techniques, making it a better fit for Reinforcement Learning from Human Feedback (RLHF).

AI models produce poor creative writing because they are trained to optimize for superficial proxies for quality, like the number of metaphors. This 'reward hacking' caters to quick judgments from human evaluators on leaderboards, mistaking flashy complexity for genuine literary taste.

While AI labs tout performance on standardized tests like math olympiads, these metrics often don't correlate with real-world usefulness or qualitative user experience. Users may prefer a model like Anthropic's Claude for its conversational style, a factor not measured by benchmarks.

The host's personal "vibe check" rankings of AI models were the inverse of the scores from an automated, LLM-judged benchmark. This highlights the gap between quantitative metrics and subjective human taste, suggesting that relying solely on AI judges misses crucial aspects of quality and real-world usability.

The best AI models are trained on data that reflects deep, subjective qualities—not just simple criteria. This "taste" is a key differentiator, influencing everything from code generation to creative writing, and is shaped by the values of the frontier lab.

For tasks where a simple right/wrong answer doesn't exist, verification is a major challenge. The solution is creating detailed rubrics with thousands of criteria, often developed with AI help. This provides a granular reward signal that allows models to climb the learning curve even in highly subjective domains.

For subjective outputs like image aesthetics and face consistency, quantitative metrics are misleading. Google's team relies heavily on disciplined human evaluations, internal 'eyeballing,' and community testing to capture the subtle, emotional impact that benchmarks can't quantify.