AI models that learn from user interactions create high switching costs. Changing providers becomes akin to firing an employee with deep institutional knowledge and onboarding a new intern, allowing AI labs to command high margins and establish a strong competitive moat.
Current AI safety proposals assume a static model is trained once and then deployed. However, models that learn continuously will require a new regulatory paradigm, such as recurring monthly or quarterly risk inspections, as one-time pre-deployment checks will become meaningless.
In a world of continual learning, the best model gets more users, which generates more data, which in turn makes the model smarter, faster. This feedback loop accelerates the leader's advantage and pressures labs to deploy models immediately, eliminating long internal testing periods.
Serving personalized, continually updated AI models is most efficient when thousands of requests are processed ('batched') at once. Large companies can achieve this, while individual users will face compute efficiency that is orders of magnitude worse, creating a strong economic advantage for large organizations.