Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

The most successful researchers at Anthropic, including senior leadership, remain deeply embedded in the details. They personally review training run progress, analyze evals, and examine underlying data. This hands-on approach is critical for developing taste and making breakthroughs in a rapidly evolving field.

Related Insights

Z.AI's culture mandates that technical leaders, including the founder, remain hands-on practitioners. The AI field evolves too quickly for a delegated, hands-off management style to be effective. Leaders must personally run experiments and engage with research to make sound, timely decisions.

AI is not a 'set and forget' solution. An agent's effectiveness directly correlates with the amount of time humans invest in training, iteration, and providing fresh context. Performance will ebb and flow with human oversight, with the best results coming from consistent, hands-on management.

Product leaders must personally engage with AI development. Direct experience reveals unique, non-human failure modes. Unlike a human developer who learns from mistakes, an AI can cheerfully and repeatedly make the same error—a critical insight for managing AI projects and team workflow.

Vercel designer Pranati Perry advises viewing AI models as interns. This mindset shifts the focus from blindly accepting output to actively guiding the AI and reviewing its work. This collaborative approach helps designers build deeper technical understanding rather than just shipping code they don't comprehend.

The essential skill for AI PMs is deep intuition, which can only be built through hands-on experimentation. This means actively using every new LLM, image, and video model upon release to objectively understand its capabilities, limitations, and trajectory, rather than relying on second-hand analysis.

Beyond standard benchmarks, Anthropic fine-tunes its models based on their "eagerness." An AI can be "too eager," over-delivering and making unwanted changes, or "too lazy," requiring constant prodding. Finding the right balance is a critical, non-obvious aspect of creating a useful and steerable AI assistant.

The key skill for an AI PM is knowing a model's current capabilities. This is built by intensely using the model and, crucially, asking it to introspect on its own unexpected behaviors to understand *why* it made a mistake, revealing gaps to fix.

The most valuable data for training enterprise AI is not a company's internal documents, but a recording of the actual work processes people use to create them. The ideal training scenario is for an AI to act like an intern, learning directly from human colleagues, which is far more informative than static knowledge bases.

Building an AI agent is the starting point, not the finish line. The real, ongoing work lies in optimizing its performance and training it on new information. This creates an essential new human-in-the-loop role focused on continuous improvement.

Anthropic's hiring philosophy prioritizes "talent density" over "talent mass." They believe a concentrated group of top AI researchers, amplified by their own frontier models, can outperform much larger teams, making elite talent and powerful models a winning combination.

Top AI Researchers Stay Hands-On, Personally Reviewing Training Runs and Evals | RiffOn