Traditional API integration requires strict adherence to a predefined contract. The new AI paradigm flips this: developers can describe their desired data format in a manifest file, and the AI handles the translation, dramatically lowering integration barriers and complexity.

The most effective integrations use external ML models as specialized scoring components within Pega's broader decisioning framework. The model's score should influence outcomes like prioritization and eligibility, but it should operate alongside, not in place of, existing business rules, eligibility criteria, and contact policies.

AI observability can be understood simply as monitoring a model's behavior for anomalies, patterns, and drifts. Like a baby monitor, it ensures the AI 'kid' stays within safe boundaries and doesn't behave unexpectedly. This constant supervision is critical for maintaining safe and predictable performance.

The primary bottleneck in improving AI is no longer data or compute, but the creation of 'evals'—tests that measure a model's capabilities. These evals act as product requirement documents (PRDs) for researchers, defining what success looks like and guiding the training process.

To evaluate an AI model, first define the business risk. Use precision when a false positive is costly (e.g., approving a faulty part). Use recall when a false negative is costly (e.g., missing a cancer diagnosis). The technical metric must align with the specific cost of being wrong.

Instead of simply swapping a model behind a stable URL, Pega's platform enables a formal release process. Using Prediction Studio's champion/challenger slots and percentage-based rollouts, teams can safely deploy, monitor, and manage new model versions. This MLOps capability turns model updates into a governed, transparent activity.

MLOps pipelines manage model deployment, but scaling AI requires a broader "AI Operating System." This system serves as a central governance and integration layer, ensuring every AI solution across the business inherits auditable data lineage, compliance, and standardized policies.

The choice of cloud provider for hosting external models (e.g., AWS SageMaker vs. Google Vertex AI) has direct consequences for which ML frameworks are supported. For example, Pega's Vertex AI integration supports XGBoost but not TensorFlow or PyTorch, unlike its broader SageMaker support. This is a critical upfront technical consideration.

The prompts for your "LLM as a judge" evals function as a new form of PRD. They explicitly define the desired behavior, edge cases, and quality standards for your AI agent. Unlike static PRDs, these are living documents, derived from real user data and are constantly, automatically testing if the product meets its requirements.