Poolside views model building as an industrialized, end-to-end process, optimizing for the speed from a researcher's idea to a trusted experimental result. This engineering-first approach uses thousands of components to streamline everything from data pipelines to training and reinforcement learning, treating models as artifacts of the process.
Poolside rejected the standard practice of packaging and copying datasets to training clusters. Instead, they stream data just-in-time. This allows training to start immediately, even while data is still materializing, and makes experimenting with data mixes a simple configuration change, dramatically accelerating the research cycle.
Initially a closed-source company, Poolside's founders chose to open source their models. This was a philosophical choice to foster a world with 100 foundation model companies instead of five, driven by the fear that a handful of companies controlling all of intelligence would feel like a dystopian sci-fi novel.
From its inception as a US company, Poolside made a conscious decision not to hire researchers in the Bay Area. Instead, they built a fully remote, global team, anticipating the intense talent war. This strategy initially slowed them down but eventually provided a competitive advantage in talent acquisition.
Poolside's Laguna S model shows significant gains from improved behaviors—like persistence, verification, and not declaring victory early—rather than just more parameters or "intelligence." This suggests that smaller models can become far more capable at knowledge work by optimizing these behavioral traits through post-training.
An unexpected benefit of building a robust, end-to-end "Model Factory" is its suitability for AI agents. These agents are now taking over tasks within the factory, such as writing code, launching jobs, and evaluating results, creating a recursive self-improvement loop for the research process itself.
Instead of relying on predefined tools passed in a system prompt, models should be given a virtual environment to write and execute code directly. This approach offers more freedom and power, as models can use loops and conditionals, moving beyond simple chained tool calls to perform complex tasks.
While frontier science may require massive models, most knowledge work relies on persistence and backtracking. As smaller models improve these behaviors, the optimal size for tasks like coding might be reached far sooner than expected, making open-source models highly competitive for a vast segment of the economy.
The distinct phase of "mid-training" exists partly because companies create dedicated teams for it, a reflection of Conway's Law. Ideally, training should be a continuous spectrum with an optimal curriculum from the start. The current staged approach is a concession to organizational design and compute limitations.
Poolside built its training codebase from scratch instead of forking a repo. While this led to painful debugging sessions, like a three-week optimizer bug, it created a resilient team with deep, non-obvious intuitions. This process revealed that blindly trusting parameters from academic papers can be misleading.
Before Poolside, Eiso Kant's startup Sourced spent years and $12M building language models for code, but failed in 2019 because "no one cared." The core idea was right, but they missed the importance of continuous scaling. The launch of ChatGPT years later felt like a personal vindication for that earlier, failed effort.
Despite its perceived complexity, the core work of building foundation models boils down to two things. Researchers are either improving the data going into the model or finding breakthroughs (in architecture, inference, etc.) that improve compute efficiency. This is a powerful, simplifying framework for a complex field.
Long training cycles mean too many variables change between models, making it impossible to attribute improvements. By training smaller models end-to-end more frequently (e.g., in weeks, not months), research teams can better understand which specific "ingredients" led to better outcomes, enabling a more scientific process.