ML Model to Production: A Complete Walkthrough

Most ML models never leave Jupyter notebooks. The ones that do often take 6 months and a dedicated platform team. Neither extreme makes sense. This post gives you the end-to-end picture: what actually happens between model.fit() and a live endpoint serving traffic.

The Four Stages Nobody Tells You About

Training a model is stage zero. Production requires four more:

1. Packaging - serializing the model and its dependencies reproducibly
2. Serving - exposing predictions via an API
3. Containerizing - making the server environment portable
4. Operating - deploying, scaling, and monitoring it

Each stage has common failure modes that kill ML projects.

Stage 1: Packaging the Model

The worst mistake is saving just the weights file. You need the full inference artifact:

pythonCopy
import joblib
import json
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Train with a pipeline that includes preprocessing
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])
pipeline.fit(X_train, y_train)

# Save the whole pipeline, not just the model
joblib.dump(pipeline, 'model_artifacts/pipeline_v1.joblib')

# Save metadata alongside
metadata = {
    "model_version": "1.0.0",
    "training_date": "2025-01-15",
    "features": ["age", "income", "tenure_months"],
    "target": "churn",
    "sklearn_version": "1.3.2",
    "python_version": "3.11",
    "train_accuracy": 0.847,
    "val_accuracy": 0.831
}
with open('model_artifacts/metadata.json', 'w') as f:
    json.dump(metadata, f, indent=2)

Why the full pipeline matters: Your StandardScaler was fit on training data. If you only save the model weights and refit the scaler on production data, your predictions will be wrong - silently wrong.

For PyTorch/TensorFlow models:

pythonCopy
import torch

# Save state dict plus architecture config
torch.save({
    'model_state_dict': model.state_dict(),
    'model_config': {
        'input_dim': 128,
        'hidden_dim': 256,
        'output_dim': 10,
        'architecture': 'FeedForward'
    },
    'training_config': {
        'epochs': 50,
        'optimizer': 'Adam',
        'lr': 1e-3
    }
}, 'model_artifacts/model_v1.pt')

Stage 2: Building the Serving Layer

FastAPI is the standard for ML APIs. Here is a complete, production-ready server:

pythonCopy
# serve.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, validator
import joblib
import numpy as np
import logging
import time
from typing import Optional

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

app = FastAPI(title="Churn Prediction API", version="1.0.0")

# Load once at startup, not per request
pipeline = joblib.load("model_artifacts/pipeline_v1.joblib")
logger.info("Model loaded successfully")

class PredictionRequest(BaseModel):
    age: float
    income: float
    tenure_months: int

    @validator('age')
    def age_must_be_positive(cls, v):
        if v <= 0 or v > 120:
            raise ValueError('age must be between 0 and 120')
        return v

    @validator('income')
    def income_must_be_non_negative(cls, v):
        if v < 0:
            raise ValueError('income must be non-negative')
        return v

class PredictionResponse(BaseModel):
    churn_probability: float
    will_churn: bool
    model_version: str = "1.0.0"
    latency_ms: Optional[float] = None

@app.get("/health")
def health_check():
    return {"status": "healthy", "model_loaded": pipeline is not None}

@app.post("/predict", response_model=PredictionResponse)
def predict(request: PredictionRequest):
    start = time.perf_counter()
    try:
        features = np.array([[request.age, request.income, request.tenure_months]])
        probability = pipeline.predict_proba(features)[0][1]
        latency_ms = (time.perf_counter() - start) * 1000

        logger.info(f"Prediction: {probability:.3f}, latency: {latency_ms:.1f}ms")
        return PredictionResponse(
            churn_probability=float(probability),
            will_churn=probability > 0.5,
            latency_ms=latency_ms
        )
    except Exception as e:
        logger.error(f"Prediction failed: {e}")
        raise HTTPException(status_code=500, detail=str(e))

Key decisions:

• Load model once at startup - not inside the endpoint function
• Validate inputs before they reach the model
• Return probability, not just a binary label - callers can pick their own threshold
• Log latency - you'll need this data later for monitoring
• Health endpoint - load balancers and Kubernetes need this

Stage 3: Containerizing

dockerfileCopy
# Dockerfile
FROM python:3.11-slim

WORKDIR /app

# Install dependencies first (better layer caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy model artifacts and server code
COPY model_artifacts/ model_artifacts/
COPY serve.py .

# Don't run as root
RUN useradd -m appuser
USER appuser

EXPOSE 8000

CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8000"]

textCopy
# requirements.txt
fastapi==0.104.1
uvicorn==0.24.0
pydantic==2.5.0
scikit-learn==1.3.2
joblib==1.3.2
numpy==1.26.2

Build and test locally:

bashCopy
docker build -t churn-predictor:1.0.0 .
docker run -p 8000:8000 churn-predictor:1.0.0

# Test it
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"age": 34, "income": 75000, "tenure_months": 24}'

Stage 4: Deploying

Option A: Docker Compose (simple, good for small scale)

yamlCopy
# docker-compose.yml
version: '3.8'
services:
  model-api:
    image: churn-predictor:1.0.0
    ports:
      - "8000:8000"
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

Option B: Kubernetes (scalable, production-grade)

yamlCopy
# k8s-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: churn-predictor
spec:
  replicas: 3
  selector:
    matchLabels:
      app: churn-predictor
  template:
    metadata:
      labels:
        app: churn-predictor
    spec:
      containers:
      - name: churn-predictor
        image: churn-predictor:1.0.0
        ports:
        - containerPort: 8000
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10

The Operational Loop

Deployment is not the end - it is the beginning of the operational lifecycle:

Train → Evaluate → Package → Deploy → Monitor → Detect drift → Retrain → ...

What to monitor from day one:

Signal	What it tells you	Alert threshold
Request latency (p50, p95, p99)	Serving performance	p99 > 500ms
Prediction distribution	Is output shifting?	KL divergence > 0.1
Input feature means/stdev	Is input data shifting?	>2σ from training baseline
Error rate (5xx)	Is the server crashing?	>0.1%
Throughput	Are you getting traffic?	Sudden drop

Common Production Failures

1. The dependency mismatch: Your local scikit-learn is 1.3.2 but the server runs 1.2.1. The model won't load. Fix: pin versions in requirements.txt, build in Docker from the start.

2. The feature order bug: You train on [age, income, tenure] but the production API receives JSON keys in alphabetical order [age, income, tenure] on Monday and [income, age, tenure] on Tuesday depending on how the upstream service serializes. Fix: always explicitly order features by name in your serving code.

3. The cold start: Your model loads slowly (transformer models can take 30+ seconds). The first request after deployment times out. Fix: warm up the model during container startup before marking the health check as passing.

4. The silent failure: Predictions are being returned but they are wrong because of a preprocessing bug. No exception is raised. Fix: log a sample of raw inputs and outputs, alert on prediction distribution shifts.

The Minimum Viable MLOps Stack

For a startup or small team:

• Model registry: MLflow (local) or the simplest object storage (S3/GCS with a naming convention)
• Serving: FastAPI in Docker
• Deployment: Docker Compose → Fly.io → Kubernetes when you actually need it
• Monitoring: Structured JSON logs → Grafana dashboard or Datadog
• CI/CD: GitHub Actions to rebuild and redeploy on merge to main

Do not build infrastructure for scale you do not have. A model serving 1000 requests per day does not need Kubernetes. Start simple, add complexity when the problem demands it.