The 2017 "Attention Is All You Need" paper, written by eight Google researchers, laid the groundwork for modern LLMs. In a striking example of the innovator's dilemma, every author left Google within a few years to start or join other AI companies, representing a massive failure to retain pivotal talent at a critical juncture.
Related Insights
Google's culture has become slow and risk-averse, not due to a lack of talent, but because its cushy compensation packages discourage top employees from leaving. This fosters an environment where talented individuals are incentivized to take fewer risks, hindering bold innovation, particularly in the fast-moving AI space.
The intense talent war in AI is hyper-concentrated. All major labs are competing for the same cohort of roughly 150-200 globally-known, elite researchers who are seen as capable of making fundamental breakthroughs, creating an extremely competitive and visible talent market.
Top AI labs face a difficult talent problem: if they restrict employee equity liquidity, top talent leaves for higher salaries. If they provide too much liquidity, newly-wealthy researchers leave to found their own competing startups, creating a constant churn that seeds the ecosystem with new rivals.
Quora's initial engineering team was a legendary concentration of talent that later dispersed to found or lead major AI players, including Perplexity and Scale AI. This highlights how talent clusters from one generation of startups can become the founding diaspora for the next.
The "Attention is All You Need" paper's key breakthrough was an architecture designed for massive scalability across GPUs. This focus on efficiency, anticipating the industry's shift to larger models, was more crucial to its dominance than the attention mechanism itself.
The "golden era" of big tech AI labs publishing open research is over. As firms realize the immense value of their proprietary models and talent, they are becoming as secretive as trading firms. The culture is shifting toward protecting IP, with top AI researchers even discussing non-competes, once a hallmark of finance.
Google authored the seminal 'Transformers' AI paper but failed to capitalize on it, allowing outsiders to build the next wave of AI. This shows how incumbents can be so 'lost in the sauce' of their current paradigm that they don't notice when their own research creates a fundamental shift.
Meta's chief AI scientist, Yann LeCun, is reportedly leaving to start a company focused on "world models"—AI that learns from video and spatial data to understand cause-and-effect. He argues the industry's focus on LLMs is a dead end and that his alternative approach will become dominant within five years.
The frenzied competition for the few thousand elite AI scientists has created a culture of constant job-hopping for higher pay, akin to a sports transfer season. This instability is slowing down major scientific progress, as significant breakthroughs require dedicated teams working together for extended periods, a rarity in the current environment.
Cohere's CEO believes if Google had hidden the Transformer paper, another team would have created it within 18 months. Key ideas were already circulating in the research community, making the discovery a matter of synthesis whose time had come, rather than a singular stroke of genius.