Counterintuitively, GitHub discovered that training coding models on more private enterprise codebases (e.g., modern web frameworks) provides little benefit. The significant performance gains come from training on scarce, legacy code like COBOL, where public data is limited but enterprise demand for modernization is high.
The industry has already exhausted the public web data used to train foundational AI models, a point underscored by the phrase "we've already run out of data." The next leap in AI capability and business value will come from harnessing the vast, proprietary data currently locked behind corporate firewalls.
Despite the hype, LinkedIn found that third-party AI tools for coding and design don't work out-of-the-box on their complex, legacy stack. Success requires deep customization, re-architecting internal platforms for AI reasoning, and working in "alpha mode" with vendors to adapt their tools.
AI coding assistants struggle with deep kernel work (CUDA, PTX) because there's little public code to learn from. Furthermore, debugging AI-generated parallel code is extremely difficult because the developer lacks the original mental model, making it less efficient than writing it themselves.
Enterprises are trapped by decades of undocumented code. Rather than ripping and replacing, agentic AI can analyze and understand these complex systems. This enables redesign from the inside out and modernizes the core of the business, bridging the gap between business and IT.
The initial magic of GitHub's Copilot wasn't its accuracy but its profound understanding of natural language. Early versions had a code completion acceptance rate of only 20%, yet the moments it correctly interpreted human intent were so powerful they signaled a fundamental technology shift.
For consumer products like ChatGPT, models are already good enough for common queries. However, for complex enterprise tasks like coding, performance is far from solved. This gives model providers a durable path to sustained revenue growth through continued quality improvements aimed at professionals.
Customizing a base model with proprietary data is only effective if a company possesses a massive corpus. At least 10 billion high-quality tokens are needed *after* aggressive deduplication and filtering. This high threshold means the strategy is only viable for the largest corporations, a much higher bar than most businesses realize.
Enterprises are finding immediate, high return on investment by using AI to port legacy codebases (like COBOL) to modern languages. This mundane task offers a 2x speed-up over traditional methods, unlocking significant infrastructure savings and even driving new developer hiring.
While AI coding assistants appear to boost output, they introduce a "rework tax." A Stanford study found AI-generated code leads to significant downstream refactoring. A team might ship 40% more code, but if half of that increase is just fixing last week's AI-generated "slop," the real productivity gain is much lower than headlines suggest.
Misha Laskin, CEO of Reflection AI, states that large enterprises turn to open source models for two key reasons: to dramatically reduce the cost of high-volume tasks, or to fine-tune performance on niche data where closed models are weak.