Measuring AI's impact by output metrics like 'percent of agent-written code' or 'number of PRs merged' is a trap. These metrics say nothing about value. Instead, focus on counterbalance metrics that measure quality and meaningful impact, such as a reduction in bugs or positive user feedback.

To maintain high velocity with AI coding assistants, Chris Fregly has stopped line-by-line code reviews and traditional unit testing. He now focuses on high-level evaluations and 'correctness harnesses' that continuously run in the background, shifting quality control from process (review) to outcome (performance).

AI agents can generate and merge code at a rate that far outstrips human review. While this offers unprecedented velocity, it creates a critical challenge: ensuring quality, security, and correctness. Developing trust and automated validation for this new paradigm is the industry's next major hurdle.

AI can generate code that passes initial tests and QA but contains subtle, critical flaws like inverted boolean checks. This creates 'trust debt,' where the system seems reliable but harbors hidden failures. These latent bugs are costly and time-consuming to debug post-launch, eroding confidence in the codebase.

With AI generating code, a developer's value shifts from writing perfect syntax to validating that the system works as intended. Success is measured by outcomes—passing tests and meeting requirements—not by reading or understanding every line of the generated code.

AI excels at generating code, making that task a commodity. The new high-value work for engineers is "verification”—ensuring the AI's output is not just bug-free, but also valuable to customers, aligned with business goals, and strategically sound.

A new paradigm for AI-driven development is emerging where developers shift from meticulously reviewing every line of generated code to trusting robust systems they've built. By focusing on automated testing and review loops, they manage outcomes rather than micromanaging implementation.

It's infeasible for humans to manually review thousands of lines of AI-generated code. The abstraction of review is moving up the stack. Instead of checking syntax, developers will validate high-level plans, two-sentence summaries, and behavioral outcomes in a testing environment.

AI tools can generate vast amounts of verbose code on command, making metrics like 'lines of code' easily gameable and meaningless for measuring true engineering productivity. This practice introduces complexity and technical debt rather than indicating progress.