Hudson River Trading shifted from handcrafted features based on human intuition to training models on raw, internet-scale market data. This emergent approach, similar to how ChatGPT is trained, has entirely overtaken traditional quant methods that relied on simpler techniques like linear regression.
Related Insights
The turning point came when a simple OpenAI API call solved a customer's problem more effectively than their complex, slow data science script. This stark contrast revealed the massive opportunity in leveraging modern AI and triggered their pivot.
The "bitter lesson" in AI research posits that methods leveraging massive computation scale better and ultimately win out over approaches that rely on human-designed domain knowledge or clever shortcuts, favoring scale over ingenuity.
High-frequency trading firms are expanding into medium-frequency horizons (days to weeks). They use their sophisticated short-term AI models, which can predict optimal prices within the next hour, to inform the execution strategy for their longer-term positions, creating a cascading effect where intraday precision enhances multi-day trading performance.
To create a truly innovative AI, Bridgewater established its "artificial investor" as a separate venture. This prevented the AI from simply inheriting the biases and flaws of the existing human-driven system. The goal was for the AI to develop its own independent, uncorrelated ideas rather than becoming a digital copy of Bridgewater itself.
Cliff Asnes explains that integrating machine learning into investment processes involves a crucial trade-off. While AI models can identify complex, non-linear patterns that outperform traditional methods, their inner workings are often uninterpretable, forcing a departure from intuitively understood strategies.
AI models can predict short-term stock prices, defying the efficient market hypothesis. However, the predictions are only marginally better than random, with an accuracy akin to "50.1%". The profitability comes not from magic, but from executing this tiny statistical edge millions of times across the market.
Beyond automating data collection, investment firms can use AI to generate novel analytical frameworks. By asking AI to find new ways to plot and interpret data inputs, the team moves from rote data entry to higher-level analysis, using the technology as a creative and strategic partner.
Demanding interpretability from AI trading models is a fallacy because they operate at a superhuman level. An AI predicting a stock's price in one minute is processing data in a way no human can. Expecting a simple, human-like explanation for its decision is unreasonable, much like asking a chess engine to explain its moves in prose.
Contrary to the hype around alternative data, the most crucial input for intraday trading AI is standard market data feeds from exchanges. This raw, high-volume data on quotes and trades is the truest expression of market intent, far outweighing the predictive value of news or social media feeds.
MDT deliberately avoids competing on acquiring novel, expensive datasets (informational edge). Instead, they focus on their analytical edge: applying sophisticated machine learning tools to long-history, high-quality standard datasets like financials and prices to find differentiated insights.
