Milestone Gate 1: ML Core Transition

You have spent weeks writing training loops, tuning hyperparameters, and reading about bias-variance trade-offs. But how do you know when you have genuinely crossed the threshold from "following tutorials" to "building ML systems that actually work"? That is the question this checkpoint is designed to answer.

This gate is not a quiz. It is a structured self-assessment rubric that forces you to test your knowledge against concrete, job-representative scenarios. Work through it honestly.

The Four Pillars of ML Core Readiness

Pillar 1: Can You Write a Training Loop From Scratch?

Not copy-paste. From scratch, with full awareness of what each line does.

pythonCopy
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset

# Can you write this without looking it up?
def train_one_epoch(model, loader, optimizer, criterion, device):
    model.train()
    total_loss = 0.0
    for X_batch, y_batch in loader:
        X_batch, y_batch = X_batch.to(device), y_batch.to(device)
        optimizer.zero_grad()
        preds = model(X_batch)
        loss = criterion(preds, y_batch)
        loss.backward()
        optimizer.step()
        total_loss += loss.item() * len(X_batch)
    return total_loss / len(loader.dataset)

Self-check questions:

• Why does optimizer.zero_grad() come before forward pass, not after?
• What happens if you call model.eval() inside this function?
• What does loss.item() do and why use it instead of loss?

If you needed to look up any of these, that is your study signal.

Pillar 2: Can You Evaluate a Model Without Leaking the Future?

Evaluation errors are the most common source of false confidence in ML systems. Test yourself:

pythonCopy
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
import numpy as np

X, y = np.random.randn(1000, 10), (np.random.randn(1000) > 0).astype(int)

# THIS IS WRONG - can you spot why?
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2)

# THIS IS RIGHT
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)  # transform only, no fit

model = LogisticRegression()
model.fit(X_train, y_train)
print(classification_report(y_test, model.predict(X_test)))

The wrong version scales using statistics from the test set, which leaks information across the split boundary and inflates your reported accuracy. This pattern kills production models.

Pillar 3: Can You Explain Bias-Variance Without Jargon?

Write a two-sentence explanation to a non-ML engineer colleague. Then check: did you use the words "underfitting," "overfitting," "training error," and "generalization error" correctly in context? Can you draw the validation curve showing where each failure mode appears?

More importantly: given a model that performs well on training data but poorly on validation data, what is your next move? There is no single right answer - but there is a structured diagnostic approach involving learning curves, regularization, and more data.

Pillar 4: Can You Detect and Fix Data Leakage?

pythonCopy
import pandas as pd
from sklearn.model_selection import TimeSeriesSplit

# Common mistake: random split on time-series data
df = pd.DataFrame({
    'date': pd.date_range('2023-01-01', periods=365),
    'feature': np.random.randn(365),
    'target': np.random.randn(365)
})

# WRONG for time-series
from sklearn.model_selection import train_test_split
train, test = train_test_split(df, test_size=0.2, random_state=42)

# RIGHT: respect temporal ordering
tscv = TimeSeriesSplit(n_splits=5)
for fold, (train_idx, val_idx) in enumerate(tscv.split(df)):
    print(f"Fold {fold}: train {len(train_idx)}, val {len(val_idx)}")

The Checklist

Score each item: 3 = confident, 2 = shaky, 1 = need to revisit.

Skill	Score
Write training loop without reference
Explain optimizer.zero_grad() purpose
Scale correctly across train/test split
Identify label leakage in a pipeline
Explain bias-variance to a non-ML engineer
Choose between accuracy, F1, and AUC for a given problem
Plot and interpret a learning curve
Implement cross-validation correctly
State two regularization techniques and when to use each
Describe early stopping and when it helps

Total 24–30: You are ready to advance to model architecture depth. Total 16–23: Targeted review needed - see the guidance below. Total under 16: Spend two more weeks on ML Foundations before moving on.

If You Are Stuck on Specific Items

• Training loop mechanics: Read linear-algebra-gradient-descent and work through the PyTorch 60-minute blitz.
• Evaluation and leakage: Read leakage-proof-feature-pipelines and evaluation-metrics-error-analysis-systems.
• Statistical reasoning: Read applied-stats-engineering-decisions.
• Pipeline structure: Read training-pipelines-experiment-strategy.

Common Mistakes at This Stage

The most common mistake is "tutorial completionism" - finishing courses and notebooks without testing whether you can reproduce the core logic from memory. Completion is not the same as fluency. The second most common mistake is skipping error analysis: understanding why a model gets specific examples wrong is more valuable than tuning hyperparameters blind.

Where to Go Next

• leakage-proof-feature-pipelines - solidify pipeline correctness before going deeper
• evaluation-metrics-error-analysis-systems - build systematic evaluation thinking
• training-pipelines-experiment-strategy - structure your experiments for reproducibility