Developers using OpenAI's API are warned that Sam Altman will analyze their usage data to identify and build competing features. This follows the classic playbook of platform owners like Microsoft and Facebook who studied third-party developers to absorb the most valuable use cases.

A key disincentive for open-sourcing frontier AI models is that the released model weights contain residual information about the training process. Competitors could potentially reverse-engineer the training data set or proprietary algorithms, eroding the creator's competitive advantage.

Despite processing 15 million clinical charts, Datycs doesn't use this data for model training. Their agreements explicitly respect that data belongs to the patient and the client—an ethical choice that prevents them from building large, aggregated language models from customer data.

Enterprise SaaS companies (the 'henhouse') should be cautious when partnering with foundation model providers (the 'fox'). While offering powerful features, these models have a core incentive to consume proprietary data for training, potentially compromising customer trust, data privacy, and the incumbent's long-term competitive moat.

To practice responsible AI, enterprises must proactively audit the 'nutrition label' of the models they use—specifically how the training data was sourced and licensed. Choosing models trained on fully licensed content is a key design principle for ensuring commercial safety and IP protection from the ground up.

Microsoft's case management AI avoids training directly on private customer data. Instead, it operates on a "bring your own knowledge" model, using only the knowledge articles and resources explicitly provided by the customer. This approach sidesteps major privacy and data governance concerns common in enterprise AI adoption.

With public data exhausted, AI companies are seeking proprietary datasets. After being rejected by established firms wary of sharing their 'crown jewels,' these labs are now acquiring the codebases of failed startups for tens of thousands of dollars as a novel source of high-quality training data.

The choice between open and closed-source AI is not just technical but strategic. For startups, feeding proprietary data to a closed-source provider like OpenAI, which competes across many verticals, creates long-term risk. Open-source models offer "strategic autonomy" and prevent dependency on a potential future rival.

Companies are becoming wary of feeding their unique data and customer queries into third-party LLMs like ChatGPT. The fear is that this trains a potential future competitor. The trend will shift towards running private, open-source models on their own cloud instances to maintain a competitive moat and ensure data privacy.