We scan new podcasts and send you the top 5 insights daily.
The auto-regressive, next-token-prediction nature of current LLMs is a 'really, really weird way to produce stuff.' True human creativity and writing insight involve knowing precisely when to make an unpredictable, non-obvious move. This is directly contrary to the model's core process, which is a slave to its immediate context and favors predictable outputs.
According to Demis Hassabis, LLMs feel uncreative because they only perform pattern matching. To achieve true, extrapolative creativity like AlphaGo's famous 'Move 37,' models must be paired with a search component that actively explores new parts of the knowledge space beyond the training data.
AI struggles with true creativity because it's designed to optimize for correctness, like proper grammar. Humans, in contrast, optimize for meaning and emotional resonance. This is why ChatGPT would not have generated Apple's iconic "Think Different" slogan—it breaks grammatical rules to create a more powerful idea. Over-reliance on AI risks losing an authentic, human voice.
The critique that LLMs lack true creativity because they only recombine and predict existing data is challenged by the observation that human creativity, particularly in branding and marketing, often operates on the exact same principles. The process involves combining existing concepts in novel ways to feel fresh, much like an LLM.
True creative mastery emerges from an unpredictable human process. AI can generate options quickly but bypasses this journey, losing the potential for inexplicable, last-minute genius that defines truly great work. It optimizes for speed at the cost of brilliance.
An LLM's core function is predicting the next word. Therefore, when it encounters information that defies its prediction, it flags it as surprising. This mechanism gives it an innate ability to identify "interesting" or novel concepts within a body of text.
AI models operate in a 'probability space,' making predictions by interpolating from past data. True human creativity operates in a 'possibility space,' generating novel ideas that have no precedent and cannot be probabilistically calculated. This is why AI can't invent something truly new.
AI excels at replicating patterns from its training data. However, top-tier authors provide value by subverting expectations and introducing surprising connections—a skill rooted in creative, pattern-breaking thought that AI struggles with. The act of writing is the act of thinking, which can't be outsourced.
AI models produce poor creative writing because they are trained to optimize for superficial proxies for quality, like the number of metaphors. This 'reward hacking' caters to quick judgments from human evaluators on leaderboards, mistaking flashy complexity for genuine literary taste.
AI models are trained on vast datasets of existing knowledge. Like a librarian who has read every book, their answers represent an average of what they have 'read.' This makes AI an aggregator of existing ideas, not a generator of truly novel, outlier concepts.
Using AI to overcome writer's block is a mistake because it aggregates existing data to provide the most popular response, which is the opposite of original thinking. True creativity comes from exploring wrong turns and unexpected paths.