ML Debugging and Iteration Loops

Software engineers have strong debugging instincts: read the stack trace, add logging, isolate the failing component. ML debugging uses the same systematic approach but the failure modes are different - the code runs fine, the model is just wrong, and figuring out why requires a structured investigation.

The ML Debugging Stack

When a model underperforms, work through this stack in order (don't skip levels):

Level 1: Is the evaluation correct?
  → Check train/test split, verify no leakage, confirm metric implementation
Level 2: Is the data correct?
  → Check label distribution, feature distributions, null rates, class balance
Level 3: Is the model training correctly?
  → Check loss curves, learning rate, overfitting vs. underfitting
Level 4: Are the features right?
  → Check feature importance, permutation importance, correlation with label
Level 5: Is the model capacity appropriate?
  → Learning curves - does more data help? Does a simpler model do nearly as well?

Most debugging time is wasted at levels 3-5 when the problem was actually at level 1 or 2. Start at the top.

Structured Error Analysis: Read Your Mistakes

pythonCopy
import pandas as pd
import numpy as np

def build_error_analysis_df(
    X_val: pd.DataFrame, y_val: pd.Series, model
) -> pd.DataFrame:
    y_pred = model.predict(X_val)
    y_prob = model.predict_proba(X_val)[:, 1]

    df = X_val.copy()
    df['true_label'] = y_val.values
    df['pred_label'] = y_pred
    df['pred_prob'] = y_prob
    df['correct'] = df['true_label'] == df['pred_label']
    df['error_type'] = np.where(
        ~df['correct'] & (df['true_label'] == 1), 'FN',
        np.where(~df['correct'] & (df['true_label'] == 0), 'FP', 'correct')
    )
    return df

error_df = build_error_analysis_df(X_val, y_val, model)

# Profile the false negatives
fn_df = error_df[error_df['error_type'] == 'FN']
print("False negative profile:")
print(fn_df[['age', 'account_age_days', 'total_spend']].describe())

# Profile the false positives
fp_df = error_df[error_df['error_type'] == 'FP']
print("\nFalse positive profile:")
print(fp_df[['age', 'account_age_days', 'total_spend']].describe())

Read 20–50 individual false negatives and false positives. You will find patterns - specific feature ranges, edge cases, or data quality issues - that aggregate statistics never reveal.

Ablation Studies: Which Components Actually Help?

pythonCopy
import copy
from sklearn.metrics import roc_auc_score

def ablation_study(base_features: list[str], X_train, y_train, X_val, y_val):
    """Test the impact of removing each feature group."""
    results = {}

    # Baseline: all features
    base_model = train_model(X_train[base_features], y_train)
    results['baseline'] = roc_auc_score(y_val, base_model.predict_proba(X_val[base_features])[:, 1])

    # Remove one feature at a time
    for feature in base_features:
        ablated_features = [f for f in base_features if f != feature]
        model = train_model(X_train[ablated_features], y_train)
        auc = roc_auc_score(y_val, model.predict_proba(X_val[ablated_features])[:, 1])
        results[f'minus_{feature}'] = auc
        print(f"Removing {feature}: AUC = {auc:.4f} (delta: {auc - results['baseline']:+.4f})")

    return results

A feature whose removal drops AUC by 0.03 is a critical feature - investigate whether it might be leaking information. A feature whose removal improves AUC is adding noise - remove it.

Label Error Analysis

Label errors are common and often the root cause of unexplained model underperformance:

pythonCopy
# Identify likely mislabeled examples using Cleanlab
# pip install cleanlab
from cleanlab.classification import CleanLearning
from sklearn.linear_model import LogisticRegression

cl = CleanLearning(LogisticRegression())
cl.fit(X_train, y_train)

label_issues = cl.get_label_issues()
likely_mislabeled = label_issues[label_issues['is_label_issue'] == True]
print(f"Found {len(likely_mislabeled)} likely mislabeled examples out of {len(X_train)}")

# Inspect the worst offenders
print(likely_mislabeled.sort_values('label_quality', ascending=True).head(20))

In real production datasets, 2–8% of labels are typically incorrect. Fixing even 1% of labels in a 100K training set can improve AUC by 0.01–0.03.

The Iteration Loop

Effective ML iteration is hypothesis-driven, not random:

1. Observe: what does the error analysis tell you?
2. Hypothesize: "FN examples have unusually low purchase frequency - the model is penalizing low-value users too harshly"
3. Experiment: add a feature that captures purchase recency vs. frequency separately
4. Measure: does CV AUC improve? Does the FN rate for low-purchase users decrease?
5. Commit: if yes, keep it. If no, roll back and form a new hypothesis.

Never add two changes at once. You will not know which one worked. One hypothesis, one change, one measurement.

Common Mistakes and Bad Instincts

Retraining without understanding the previous failure. If your model underperforms, figure out why before collecting more data or changing architectures. More data does not fix a label error problem. A different architecture does not fix a leakage problem.

Measuring improvement on training data. If you tune based on training performance, you are overfitting your experimental loop to the training distribution. Always measure on a held-out validation set that you have not touched during the iteration.

Reading 2 error examples instead of 50. Two examples show noise. Fifty examples start to show patterns. Always read enough errors to distinguish systematic patterns from random variation.

Where to Go Next

• Module 9 (sklearn Pipelines and Reproducibility) covers how to structure these iteration loops so every experiment is reproducible and comparable.
• Module 7 (Evaluation) covers choosing the right metric to measure what you are debugging toward.