Data Visualization and Exploratory Analysis With Judgment

EDA Is Investigation, Not Decoration

Exploratory data analysis has two failure modes. The first: no EDA at all - diving straight into model training without understanding the data. The second: random charts that look thorough but answer no specific questions.

Good EDA is hypothesis-driven investigation. You generate specific questions ("is the label distribution balanced?" "do any features correlate suspiciously strongly with the label?"), build visualizations that answer them, and document what you learn for modeling decisions downstream.

This module shows EDA as that disciplined investigation.

Start With Questions, Not Charts

Before opening a notebook, write down the questions your EDA must answer:

1. What does the distribution of each feature look like? (Outliers, skewness, unexpected values)
2. Is the target variable balanced? (Class imbalance affects metric choice and training)
3. Are any features correlated with the target? (Signal strength, unexpected leakage)
4. Are any features highly correlated with each other? (Multicollinearity, redundancy)
5. Are there signs of data leakage? (Features that correlate too strongly with the label)
6. Does the data look different in different time periods or subgroups? (Distribution shift)

Every chart you produce should answer one of these questions. If it does not, it is decoration.

Distributions: Single Features

pythonCopy
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv("data/processed/features.csv")

fig, axes = plt.subplots(2, 3, figsize=(14, 8))
numeric_cols = ["age", "income", "tenure_days", "num_transactions", "avg_spend", "recency"]

for ax, col in zip(axes.flatten(), numeric_cols):
    ax.hist(df[col].dropna(), bins=50, edgecolor='none')
    ax.set_title(col)
    ax.set_xlabel("")
    null_pct = df[col].isna().mean()
    if null_pct > 0:
        ax.set_xlabel(f"{null_pct:.1%} missing")

plt.tight_layout()

Look for:

• Extreme skewness: Does income span six orders of magnitude? Log-transform may help.
• Suspicious spikes: A spike at 0 may mean "missing, imputed as 0" rather than a real value.
• Truncation: A hard cutoff at a round number (100, 1000) may indicate a system limit or data collection artifact.
• Bimodality: Two distinct humps often indicate two different populations mixed together.

Target Variable Analysis

pythonCopy
# Class distribution for classification
label_counts = df["label"].value_counts()
print(label_counts)
print(f"\nPositive class rate: {label_counts[1] / len(df):.3%}")

# Plot
fig, axes = plt.subplots(1, 2, figsize=(10, 4))
label_counts.plot(kind="bar", ax=axes[0])
axes[0].set_title("Class distribution (counts)")

label_counts.plot(kind="pie", autopct='%1.1f%%', ax=axes[1])
axes[1].set_title("Class distribution (%)")
plt.tight_layout()

A 95/5 positive-class split means:

• Accuracy is a misleading metric (a model that always predicts 0 gets 95%)
• You should use AUC-ROC, PR-AUC, or F1 as your primary evaluation metric
• You may need class-weighted loss or sampling strategies during training

Feature - Target Relationships: Signal and Leakage

pythonCopy
# For numeric features: compare distributions across classes
fig, axes = plt.subplots(2, 3, figsize=(14, 8))

for ax, col in zip(axes.flatten(), numeric_cols):
    for label, group in df.groupby("label"):
        ax.hist(group[col].dropna(), bins=30, alpha=0.6, label=f"label={label}")
    ax.set_title(col)
    ax.legend()

plt.tight_layout()

Strong separation between the blue and orange distributions means a feature has predictive power. Perfect separation is a red flag - it may indicate leakage (the feature is derived from or caused by the label).

pythonCopy
# Point-biserial correlation: numeric feature vs. binary label
from scipy.stats import pointbiserialr

correlations = {}
for col in numeric_cols:
    valid = df[[col, "label"]].dropna()
    corr, pval = pointbiserialr(valid["label"], valid[col])
    correlations[col] = {"correlation": corr, "p_value": pval}

corr_df = pd.DataFrame(correlations).T.sort_values("correlation", key=abs, ascending=False)
print(corr_df)

Any correlation above 0.8–0.9 should be investigated immediately. Ask: "Would I actually have this feature at prediction time?"

Correlation Between Features: Redundancy and Multicollinearity

pythonCopy
# Correlation matrix
corr_matrix = df[numeric_cols].corr()

plt.figure(figsize=(10, 8))
sns.heatmap(
    corr_matrix,
    annot=True,
    fmt=".2f",
    cmap="RdBu_r",
    center=0,
    square=True,
)
plt.title("Feature correlation matrix")

High correlation between two features (|r| > 0.9) means they carry similar information. Including both can cause:

• Instability in linear models (the coefficients become sensitive to small data changes)
• Wasted compute in tree models (they will learn similar splits)

Consider dropping one, combining them (ratio, difference), or letting L1 regularization select.

Time-Based Patterns: Detecting Distribution Shift

pythonCopy
# Is the data different over time?
df["date"] = pd.to_datetime(df["event_date"])
df["month"] = df["date"].dt.to_period("M")

monthly_stats = df.groupby("month").agg(
    n_records=("label", "count"),
    positive_rate=("label", "mean"),
    avg_income=("income", "mean"),
).reset_index()
monthly_stats["month"] = monthly_stats["month"].astype(str)

fig, axes = plt.subplots(3, 1, figsize=(12, 9), sharex=True)
for ax, col in zip(axes, ["n_records", "positive_rate", "avg_income"]):
    ax.plot(monthly_stats["month"], monthly_stats[col], marker='o')
    ax.set_ylabel(col)
    ax.tick_params(axis='x', rotation=45)
plt.tight_layout()

Trends or sudden shifts in positive_rate or feature distributions over time indicate:

• Real-world changes in user behavior (concept drift)
• Data collection changes (pipeline failures, schema migrations)
• Seasonality effects your model needs to account for

If you see a sharp drop in avg_income in March 2024, investigate before using that data for training.

Subgroup Analysis: Finding Failure-Prone Slices

pythonCopy
# Performance by device type - do features mean different things?
for device, group in df.groupby("device_type"):
    print(f"\n{device}:")
    print(group[numeric_cols].describe().round(2))

Models that perform well on average often perform poorly on specific subgroups. EDA on subgroups before training tells you:

• Which subgroups may need separate models or features
• Which subgroups have too few examples to learn from
• Where feature engineering has gaps

Documenting Your Findings

EDA findings should be documented in a structured way, not just as a notebook full of plots:

markdownCopy
## EDA Summary: Churn Prediction Dataset

**Dataset**: 145,823 user-months, Jan 2023 – Jun 2024
**Positive class rate**: 8.3% (monthly churn)

### Key findings
1. **`days_since_last_transaction` is the strongest predictor** (r = 0.42). No leakage risk - this is computable at prediction time from prior history.
2. **`total_spend` is heavily right-skewed** (99th percentile = $12,400 vs. median $87). Will log-transform before training.
3. **Mobile users churn at 2.1× the rate of desktop users**. Device type should be included as a feature; may warrant separate models.
4. **Sharp drop in `avg_income` in March 2024**  -  corresponds to a schema migration. Will exclude March 2024 data from training.
5. **`lifetime_value` correlates 0.98 with the label**  -  this is post-label leakage. Will exclude it.

### Modeling implications
- Primary metric: PR-AUC (imbalanced, 8.3% positive rate)
- Log-transform `total_spend` and `num_transactions`
- Add `income_was_missing` indicator (12.3% missing)
- Exclude `lifetime_value`
- Split at Dec 31, 2023 for temporal train/val split

This document becomes the justification for every preprocessing decision downstream.

Common Mistakes and Bad Instincts

Running EDA with no specific questions. If you are generating charts without a question each chart answers, you are doing decoration, not investigation.

Not checking for temporal drift. Static distributions look fine; monthly trends reveal the problem. Always plot key statistics over time.

Missing the leakage signal. A feature with suspiciously high correlation (> 0.9) with the target label almost always represents leakage. Investigate every feature above this threshold before including it.

Not documenting findings. EDA findings that live only in a notebook are invisible to future team members and to your future self. Write a summary with specific modeling implications.

Where to Go Next

Module 10 begins Phase 2 of the curriculum: core machine learning engineering. EDA findings from this module directly inform the problem framing, metric selection, and feature engineering decisions in Module 10's supervised learning workflow.