With agent loops automating execution, the highest-value human skill becomes designing the environment and rules for the AI. This involves writing the strategy document (like 'program.md'), defining success metrics, and constructing the evaluation function. Your job is no longer to do the work, but to architect the system in which the work gets done.

Mozilla's agent worked well because it had a definitive verification signal: a fuzzing build that clearly reports 'you win or you lose'. For projects with more ambiguous outcomes, defining a crisp, automatable success metric is a critical prerequisite for effective agentic work.

The key to enabling an AI agent like Ralph to work autonomously isn't just a clever prompt, but a self-contained feedback loop. By providing clear, machine-verifiable "acceptance criteria" for each task, the agent can test its own work and confirm completion without requiring human intervention or subjective feedback.

Unlike coding, where context is centralized (IDE, repo) and output is testable, general knowledge work is scattered across apps. AI struggles to synthesize this fragmented context, and it's hard to objectively verify the quality of its output (e.g., a strategy memo), limiting agent effectiveness.

A subtle failure mode for agentic loops is when a task is marked "done" because it met the literal finish line, but the output is bland. This isn't the agent's fault; it's a failure of the user to properly define the goal with sufficient quality criteria.

Agentic loops are not a universal solution. They are most effective in domains where success can be measured by a clear, objective score and where failed experiments are cheap and quick. This framework helps identify the best business processes to automate, starting with areas like code generation or ad testing, not subjective, slow-moving tasks like political negotiation.

Iterative AI agent loops, like Andre Karpathy's Auto Research, are not just another tool but a new foundational building block of work. Similar to how spreadsheets or email became ubiquitous across all roles and industries, these loops will be a core component of how knowledge work is performed, fundamentally changing process and productivity.

To get the best results from an AI agent, provide it with a mechanism to verify its own output. For coding, this means letting it run tests or see a rendered webpage. This feedback loop is crucial, like allowing a painter to see their canvas instead of working blindfolded.

Agentic loops excel in constrained tasks with clear feedback, like fixing code based on an AI-generated review score. They fail in open-ended creative tasks like building an application, where they make costly, incorrect assumptions about product details.