Google's Embedding 2 model is a significant infrastructure upgrade because it is 'natively multimodal.' This allows AI to directly understand and retrieve images, diagrams, and text without first converting non-text data into lossy captions. This makes internal knowledge bases and co-pilots dramatically more effective and accurate for enterprises.

Stripe avoids costly system rebuilds by treating its new payments foundation model as a modular component. Its powerful embeddings are simply added as new features to many existing ML classifiers, instantly boosting their performance with minimal engineering effort.

A unified tokenizer, while efficient, may not be optimal for both understanding and generation tasks. The ideal data representation for one task might differ from the other, potentially creating a performance bottleneck that specialized models would avoid.

Current LLMs abstract language into discrete tokens, losing rich information like font, layout, and spatial arrangement. A "pixel maximalist" view argues that processing visual representations of text (as humans do) is a more lossless, general approach that captures the physical manifestation of language in the world.

Autoencoding models (e.g., BERT) are "readers" that fill in blanks, while autoregressive models (e.g., GPT) are "writers." For non-generative tasks like classification, a tiny autoencoding model can match the performance of a massive autoregressive one, offering huge efficiency gains.

Customizing a base model with proprietary data is only effective if a company possesses a massive corpus. At least 10 billion high-quality tokens are needed *after* aggressive deduplication and filtering. This high threshold means the strategy is only viable for the largest corporations, a much higher bar than most businesses realize.

By training a smaller, specialized model where company data is in the weights, firms avoid the high token costs of repeatedly feeding context to large frontier models. This makes complex, data-intensive workflows significantly cheaper and faster.

While frontier models like Claude excel at analyzing a few complex documents, they are impractical for processing millions. Smaller, specialized, fine-tuned models offer orders of magnitude better cost and throughput, making them the superior choice for large-scale, repetitive extraction tasks.

Jazmia Henry defines her "full stack" role as a four-stage process: obsessive data curation, custom tokenizer/embedding development, model training (pre-training and RL), and finally, optimizing the trained model for efficient inference, which is often overlooked.