Anthropic strategically focuses on "vision in" (AI understanding visual information) over "vision out" (image generation). This mimics a real developer who needs to interpret a user interface to fix it, but can delegate image creation to other tools or people. The core bet is that the primary bottleneck is reasoning, not media generation.

Once models reach human-level performance via supervised learning, they hit a ceiling. The next step to achieve superhuman capabilities is moving to a Reinforcement Learning from Human Feedback (RLHF) paradigm, where humans provide preference rankings ("this is better") rather than creating ground-truth labels from scratch.

The primary driver for fine-tuning isn't cost but necessity. When applications like real-time voice demand low latency, developers are forced to use smaller models. These models often lack quality for specific tasks, making fine-tuning a necessary step to achieve production-level performance.

Basic supervised fine-tuning (SFT) only adjusts a model's style. The real unlock for enterprises is reinforcement fine-tuning (RFT), which leverages proprietary datasets to create state-of-the-art models for specific, high-value tasks, moving beyond mere 'tone improvements.'

Fine-tuning an AI model is most effective when you use high-signal data. The best source for this is the set of difficult examples where your system consistently fails. The processes of error analysis and evaluation naturally curate this valuable dataset, making fine-tuning a logical and powerful next step after prompt engineering.

For use cases demanding strict fidelity to a complex knowledge domain like Catholic theology, fine-tuning existing models proves inadequate over the long tail of user queries. This necessitates the more expensive path of training a model from scratch.

Despite base models improving, they only achieve ~90% accuracy for specific subjects. Enterprises require the 99% pixel-perfect accuracy that LoRAs provide for brand and character consistency, making it an essential, long-term feature, not a stopgap solution.

While SAM3 can act as a "tool" for LLMs, researchers argue that fundamental vision tasks like counting fingers should be a native, immediate capability of a frontier model, akin to human System 1 thinking. Relying on tool calls for simple perception indicates a critical missing capability in the core model.

The visual domain is more fertile for open-source contributions because small tweaks, like fine-tuning an aesthetic, produce tangible, distinct results. In contrast, fine-tuned LLMs often feel monolithic with less perceptible differences, leading to a less diverse open-source community.