Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

To maintain high velocity with AI coding assistants, Chris Fregly has stopped line-by-line code reviews and traditional unit testing. He now focuses on high-level evaluations and 'correctness harnesses' that continuously run in the background, shifting quality control from process (review) to outcome (performance).

Related Insights

The focus of "code review" is shifting from line-by-line checks to validating an AI's initial architectural plan. After plan approval, AI agents like OpenAI's Codex can effectively review their own generated code, a capability they have been explicitly trained for, making human code review obsolete.

As AI generates more code than humans can review, the validation bottleneck emerges. The solution is providing agents with dedicated, sandboxed environments to run tests and verify functionality before a human sees the code, shifting review from process to outcome.

Cisco is developing its AI defense product entirely with AI-written code, with human engineers acting as "spec developers." This fundamentally changes the software development lifecycle, making code review—not code creation—the primary bottleneck and indicating a future where engineering productivity is redefined.

AI tools are automating code generation, reducing the time developers spend writing it. Consequently, the primary skill shifts to carefully reviewing and verifying the AI-generated code for correctness and security. This means a developer's time is now spent more on review and architecture than on implementation.

With AI generating code, a developer's value shifts from writing perfect syntax to validating that the system works as intended. Success is measured by outcomes—passing tests and meeting requirements—not by reading or understanding every line of the generated code.

The role of a senior developer is evolving. They now focus on defining outcomes by writing tests that a piece of code must accomplish. The AI then generates the actual implementation, allowing small teams to build complex systems in a fraction of the traditional time.

AI coding agents like Amazon Q are most effective when paired with senior developers. Their primary skill shifts from writing original code to reviewing AI-generated output. This leverage turns already high-performing developers into significantly more productive leaders, as their expertise in code review becomes the new bottleneck.

A new paradigm for AI-driven development is emerging where developers shift from meticulously reviewing every line of generated code to trusting robust systems they've built. By focusing on automated testing and review loops, they manage outcomes rather than micromanaging implementation.

It's infeasible for humans to manually review thousands of lines of AI-generated code. The abstraction of review is moving up the stack. Instead of checking syntax, developers will validate high-level plans, two-sentence summaries, and behavioral outcomes in a testing environment.

As AI generates more code, the core engineering task evolves from writing to reviewing. Developers will spend significantly more time evaluating AI-generated code for correctness, style, and reliability, fundamentally changing daily workflows and skill requirements.

Top Engineers Are Abandoning Manual Code Reviews and Unit Tests for Continuous AI Evals | RiffOn