Observability, EvalOps, and Governance

Running a model in production without observability is like operating a server without logs. You will eventually have a problem, and you will have no idea what happened or when it started. The three pillars of ML observability - infrastructure metrics, data quality metrics, and model quality metrics - give you the full picture. EvalOps adds systematic model evaluation in CI. Governance ties it all together with audit trails and approval gates.

Pillar 1: Infrastructure Metrics

Infrastructure metrics tell you if your serving system is healthy. These are the same metrics you would track for any service:

• Latency: p50, p95, p99 per endpoint. Alert when p99 exceeds your SLA.
• Error rate: HTTP 5xx / total requests. Alert when > 1%.
• Throughput: requests per second. Watch for unexpected drops (upstream issue) or spikes (traffic anomaly).
• Resource utilization: CPU, memory, GPU utilization. Alert when CPU or GPU > 85% sustained.

These are table stakes. A Prometheus + Grafana stack covers them without custom instrumentation:

yamlCopy
# prometheus/alerts.yml
groups:
  - name: ml-serving
    rules:
      - alert: HighPredictionLatency
        expr: histogram_quantile(0.99, rate(http_request_duration_seconds_bucket{job="churn-model"}[5m])) > 0.2
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "p99 latency > 200ms for churn-model"

Pillar 2: Data Quality Metrics

Data quality metrics tell you if the inputs to your model look like what it was trained on. This is where ML systems diverge from standard services - the inputs themselves can degrade silently.

Key checks to run on every batch of incoming data:

• Schema validation: are all expected features present with the expected types?
• Null rate per feature: alert if null rate for a feature exceeds training baseline by more than 10 pp
• Distribution shift: compare incoming feature distributions to training set using KS test or PSI
• Volume checks: if you expect 10,000 rows per hour and receive 300, something upstream is broken

pythonCopy
import great_expectations as ge

def validate_incoming_features(df):
    suite = ge.from_pandas(df)
    suite.expect_column_to_exist("user_tenure_days")
    suite.expect_column_values_to_not_be_null("user_tenure_days", mostly=0.95)
    suite.expect_column_mean_to_be_between("user_tenure_days", min_value=60, max_value=400)
    result = suite.validate()
    if not result["success"]:
        raise DataQualityError(f"Feature validation failed: {result['statistics']}")
    return df

Pillar 3: Model Quality Metrics

Model quality metrics tell you if the model is performing well. This is the hardest pillar because ground truth labels are often delayed or unavailable.

Strategies by availability of ground truth:

Label availability	Strategy
Real-time (fraud, clicks)	Direct metric computation on sliding window
Delayed (churn, conversion)	Cohort-based metric with lag - measure M-30 performance now
Unavailable	Proxy metrics: prediction distribution, confidence scores, output entropy

For LLMs, output quality proxies include: length distribution, refusal rate, n-gram diversity, embedding distance from reference outputs, and output classifier scores.

EvalOps: Running Evaluations in CI

EvalOps is the practice of running structured model evaluations on every code change, not just before deployment. Think of it as unit tests for model behavior.

A basic CI eval stage:

yamlCopy
# .github/workflows/ml-ci.yml
- name: Run model evaluations
  run: |
    python evals/run_evals.py \
      --model-path models/churn_v3 \
      --eval-suite evals/suites/regression_suite.json \
      --threshold-file evals/thresholds.json \
      --fail-on-regression

The eval suite contains:

• Regression tests: specific inputs that must produce known outputs
• Slice evaluations: performance on demographic or behavioral subgroups
• Adversarial probes: edge cases the model should handle correctly

The --fail-on-regression flag blocks the merge if any metric drops below a defined threshold. This is how you prevent silent regressions from reaching production.

Governance: Audit Trails, Model Cards, and Approval Gates

Governance is the organizational layer that ensures model deployments are intentional, documented, and reversible.

Model cards document what a model does, what data it was trained on, known failure modes, and intended use. They are the artifact you produce before any production deployment:

markdownCopy
## Model Card: churn-prediction-v3

**Intended use**: predict 30-day churn probability for subscription users

**Training data**: user events Jan 2023 – Dec 2023, 2.1M users

**Evaluation**:
- AUC: 0.91 on holdout set
- TPR at 10% FPR: 0.74

**Known limitations**:
- Underperforms for users < 30 days tenure (n < 50,000 in training)
- Not validated for enterprise accounts

**Approval**: ML lead sign-off required before production deployment

Approval gates require a human sign-off before a model goes live. In practice this is a PR review step or a mandatory Slack/Jira approval tied to the deployment pipeline.

Audit logs record every model version deployed, by whom, when, and with what configuration. A simple implementation stores this in a model_deployments table and writes to it from your CI/CD pipeline.

How the Three Pillars Connect

Infrastructure metrics tell you when the system is broken. Data quality metrics tell you when the inputs are wrong. Model quality metrics tell you when the outputs are degrading. EvalOps catches regressions before deployment. Governance ensures every deployment is deliberate.

A failure that bypasses all three pillars simultaneously is rare. Most production incidents fail one pillar at a time: a schema change breaks data quality, an upstream outage tanks throughput, a distribution shift degrades prediction quality. If you have all three instrumented, you catch them before they compound.

Common Mistakes

Instrumenting only infrastructure. HTTP 200 does not mean the model is working correctly. You need model quality metrics to know that predictions are meaningful.

Running evals only before major releases. Regressions are introduced by small, seemingly-unrelated changes. Run evals on every merge.

Writing a model card after the fact. The model card should be drafted as part of the development process, not as a compliance exercise before deployment.

Where to Go Next

• monitoring-drift-llm-evaluation - go deep on drift detection and LLM-specific quality monitoring
• mlops-cicd-ml-systems - integrate the EvalOps patterns here into a full CI/CD pipeline
• milestone-gate-2-production-readiness - use this pillar framework to audit your current production readiness