Monitoring, Drift, and LLM Evaluation

Model performance in production is not static. The world changes, user behavior evolves, upstream systems are modified, and the distribution of inputs shifts away from what the model was trained on. Detecting this degradation early - and responding correctly - is one of the most operationally important skills in production ML.

Why Models Degrade

Degradation happens through several mechanisms:

• Feature drift: the distribution of input features changes. A user tenure feature trained on a 2022 cohort looks different for a 2024 cohort.
• Label shift: the base rate of the target changes. If churn rate drops from 8% to 3%, a model calibrated on 8% will be overconfident.
• Concept drift: the relationship between features and labels changes. Fraud patterns shift as fraudsters adapt.
• Upstream schema change: a dependency changes a field name or data type, causing a feature to be computed incorrectly or to be missing entirely.

The first three are gradual; the fourth is sudden and often catastrophic.

Feature Drift Detection

The two most common statistical tests for feature drift are the Kolmogorov-Smirnov (KS) test and the Population Stability Index (PSI).

KS test compares the cumulative distribution of a feature in production vs training. It returns a p-value and a statistic (D).

pythonCopy
from scipy import stats
import numpy as np

def ks_drift_test(reference: np.ndarray, production: np.ndarray, threshold: float = 0.05):
    stat, p_value = stats.ks_2samp(reference, production)
    drifted = p_value < threshold
    return {"statistic": round(stat, 4), "p_value": round(p_value, 4), "drifted": drifted}

PSI compares distributions binned into deciles. It is directional (production vs reference) and produces a single score:

pythonCopy
def compute_psi(reference: np.ndarray, production: np.ndarray, bins: int = 10) -> float:
    breakpoints = np.percentile(reference, np.linspace(0, 100, bins + 1))
    ref_counts = np.histogram(reference, bins=breakpoints)[0] / len(reference)
    prod_counts = np.histogram(production, bins=breakpoints)[0] / len(production)

    # Clip to avoid log(0)
    ref_counts = np.clip(ref_counts, 1e-6, None)
    prod_counts = np.clip(prod_counts, 1e-6, None)

    psi = np.sum((prod_counts - ref_counts) * np.log(prod_counts / ref_counts))
    return round(float(psi), 4)

PSI interpretation: < 0.1 = no significant change; 0.1–0.2 = moderate shift, investigate; > 0.2 = significant drift, act.

Prediction Distribution Shift

Even without feature-level drift metrics, the distribution of your model's output scores tells you a lot. If your churn model's average predicted probability was 0.12 in January and is 0.31 in March, something has changed - either the real churn rate has increased, or the model is receiving inputs it was not designed for.

pythonCopy
def monitor_score_distribution(scores: list[float], baseline_mean: float, baseline_std: float):
    mean = np.mean(scores)
    std = np.std(scores)
    z_score = abs(mean - baseline_mean) / (baseline_std / np.sqrt(len(scores)))
    return {
        "current_mean": round(mean, 4),
        "baseline_mean": round(baseline_mean, 4),
        "z_score": round(z_score, 2),
        "alert": z_score > 3.0  # 3-sigma rule
    }

LLM Output Quality Drift

LLM-based systems face a different class of drift. Inputs are free-form text; outputs are generated sequences. The standard statistical tests do not apply directly.

Practical proxies for LLM output quality:

Proxy	How to measure
Response length distribution	Alert if mean token length drops > 20% (truncation issues)
Refusal rate	Count outputs matching refusal patterns; alert if > 5%
Output embedding distance	Embed outputs, track mean cosine distance to reference outputs
Classifier score	Run a small output-quality classifier trained on human-labeled examples
Task-specific eval	For code generation: run the code; for summarization: ROUGE vs reference

An eval harness pattern for LLMs:

pythonCopy
class LLMEvalHarness:
    def __init__(self, model_client, eval_suite: list[dict]):
        self.client = model_client
        self.suite = eval_suite  # [{prompt, expected_behavior, scorer}]

    def run(self) -> dict:
        results = []
        for case in self.suite:
            output = self.client.generate(case["prompt"])
            score = case["scorer"](output)
            results.append({"case_id": case["id"], "score": score, "passed": score >= case["threshold"]})

        pass_rate = sum(r["passed"] for r in results) / len(results)
        return {"pass_rate": round(pass_rate, 3), "results": results}

Alert Thresholds and Response Levels

Not every drift signal requires the same response. A practical tiering:

Severity	Signal	Response
Info	PSI 0.05–0.10 on any feature	Log, include in weekly model health report
Warning	PSI > 0.10 on a top-10 feature	Notify on-call ML engineer, schedule investigation
Critical	PSI > 0.20 on a top-3 feature, or prediction mean shift > 3σ	Page on-call immediately, consider traffic reduction
Emergency	Prediction error rate > 10%, upstream schema failure	Activate rollback runbook

Retrain vs Update Prompt

For traditional ML models, significant drift usually calls for retraining. For LLM-based systems, the decision tree is different:

Retrain when: the underlying task relationship has changed, label distribution has shifted, or fine-tuning was used and needs to be refreshed.

Update prompt when: the model's behavior is correct given the right context, but the prompt is not eliciting that behavior. Prompt update is faster, cheaper, and reversible - try it first.

Switch or augment retrieval when: the model lacks relevant knowledge. A RAG update (adding new documents to the index) is cheaper than retraining and often sufficient.

Common Mistakes

Monitoring aggregate metrics only. A model can have stable average performance while dramatically degrading on a high-value subpopulation. Always slice by user segment, product line, or geography.

Tight alert thresholds that flood on-call. Thresholds that fire on normal variation train your team to ignore alerts. Calibrate thresholds using 30 days of historical variance before going live.

Confusing upstream failures with model drift. A feature that silently goes to null because of a schema change looks like drift but is actually a data pipeline bug. Separate your data quality alerts from your drift alerts.

Where to Go Next

• observability-evalops-governance - extend drift monitoring into a full observability + governance system
• mlops-cicd-ml-systems - automate retraining triggers based on drift signals in your CI/CD pipeline
• mlops-deployment-serving - rollback is your fastest response to a critical drift event