Even within OpenAI, a stark performance gap exists. Engineers who avoid using agentic AI for coding are reportedly 10x less productive across metrics like code volume, commits, and business impact. This creates significant challenges for performance management and HR.
Block's CTO quantifies the impact of their internal AI agent, Goose. AI-forward engineering teams save 8-10 hours weekly, a figure he considers the absolute baseline. He notes, "this is the worst it will ever be," suggesting exponential gains are coming.
The most significant productivity gains come from applying AI to every stage of development, including research, planning, product marketing, and status updates. Limiting AI to just code generation misses the larger opportunity to automate the entire engineering process.
AI coding assistants won't make fundamental skills obsolete. Instead, they act as a force multiplier that separates engineers. Great engineers use AI to become exceptional by augmenting their deep understanding, while mediocre engineers who rely on it blindly will fall further behind.
While objective studies on AI coding assistants are mixed, their enterprise ROI is easily justified. Executives approve the investment because their most valuable employees—engineers—report significant productivity gains, making the business case simple regardless of hard data.
AI acts as a massive force multiplier for software development. By using AI agents for coding and code review, with humans providing high-level direction and final approval, a two-person team can achieve the output of a much larger engineering organization.
While AI coding assistants appear to boost output, they introduce a "rework tax." A Stanford study found AI-generated code leads to significant downstream refactoring. A team might ship 40% more code, but if half of that increase is just fixing last week's AI-generated "slop," the real productivity gain is much lower than headlines suggest.
AI disproportionately benefits top performers, who use it to amplify their output significantly. This creates a widening skills and productivity gap, leading to workplace tension as "A-players" can increasingly perform tasks previously done by their less-motivated colleagues, which could cause resentment and organizational challenges.
A Meta study found expert programmers were less productive with AI tools. The speaker suggests this is because users thought they were faster while actually being distracted (e.g., social media) waiting for the AI, highlighting a dangerous gap between perceived and actual productivity.
Data on AI tool adoption among engineers is conflicting. One A/B test showed that the highest-performing senior engineers gained the biggest productivity boost. However, other companies report that opinionated senior engineers are the most resistant to using AI tools, viewing their output as subpar.
AI tools can generate vast amounts of verbose code on command, making metrics like 'lines of code' easily gameable and meaningless for measuring true engineering productivity. This practice introduces complexity and technical debt rather than indicating progress.