Interview Readiness for ML/AI Engineering Roles

ML engineering interviews test a different mix than standard SWE interviews. You will face ML theory questions, ML system design, standard coding, and behavioral interviews all in the same loop. The preparation strategy is different: you cannot brute-force your way through 500 LeetCode problems and expect to pass. You need depth in both ML concepts and engineering execution.

The Interview Loop Structure

Most ML engineering interviews have four components:

Round	What's tested	How to prepare
ML fundamentals	Bias/variance, gradient descent, regularization, metrics	Conceptual flashcards + explain-aloud practice
ML coding	Implement a model or algorithm from scratch	Build without sklearn: logistic regression, k-means, backprop
ML system design	Design a recommendation system, search, or LLM feature	Framework + 5-6 case studies
Behavioral	Project impact, failures, collaboration	STAR format for 5-6 strong stories

Standard SWE coding (algorithms and data structures) is also present - treat that as a prerequisite solved separately.

ML Fundamentals: The Core Questions

Practice explaining these out loud until they are fluent:

Bias-Variance Tradeoff:

"Bias is error from overly simplistic assumptions in the learning algorithm - a high-bias model underfits. Variance is error from sensitivity to small fluctuations in the training set - a high-variance model overfits. Regularization, dropout, and cross-validation address high variance. More expressive models, more data, and feature engineering address high bias. In practice, I monitor both train and val loss: if train loss is high, I am biased; if val-train gap is large, I have high variance."

Why does gradient descent not always find the global minimum?

"For non-convex loss landscapes (all neural networks), gradient descent can get stuck in local minima or saddle points. In practice, for large overparameterized networks, most local minima are nearly as good as the global minimum - the problem is saddle points, which SGD's noise helps escape. This is an empirical observation, not a guarantee."

When would you use precision vs. recall as your primary metric?

"Precision when false positives are costly - spam classification, where a false positive (legitimate email marked as spam) is highly disruptive. Recall when false negatives are costly - cancer screening, where missing a true positive has severe consequences. F1 is a balanced metric; PR-AUC is better than ROC-AUC when classes are imbalanced."

ML Coding: What to Practice

Implement from scratch (without sklearn):

pythonCopy
# Logistic regression with gradient descent
import numpy as np

class LogisticRegression:
    def __init__(self, lr=0.01, n_iter=1000, reg_lambda=0.01):
        self.lr = lr
        self.n_iter = n_iter
        self.reg_lambda = reg_lambda
        self.weights = None
        self.bias = None

    def fit(self, X: np.ndarray, y: np.ndarray):
        n_samples, n_features = X.shape
        self.weights = np.zeros(n_features)
        self.bias = 0.0

        for _ in range(self.n_iter):
            z = X @ self.weights + self.bias
            y_hat = 1 / (1 + np.exp(-z))

            dw = (X.T @ (y_hat - y)) / n_samples + self.reg_lambda * self.weights
            db = (y_hat - y).mean()

            self.weights -= self.lr * dw
            self.bias -= self.lr * db

    def predict_proba(self, X: np.ndarray) -> np.ndarray:
        z = X @ self.weights + self.bias
        return 1 / (1 + np.exp(-z))

    def predict(self, X: np.ndarray, threshold=0.5) -> np.ndarray:
        return (self.predict_proba(X) >= threshold).astype(int)

Other algorithms to implement from scratch:

• K-means clustering
• Decision tree (Gini impurity split)
• One backprop step through a two-layer network
• Cosine similarity and BM25 scoring

ML System Design: The Framework

When given "design a recommendation system" or "design a search ranking system," use this structure:

1. CLARIFY (2-3 min)
   - What are the inputs? (user, context, query)
   - What are we optimizing? (CTR, revenue, engagement)
   - Scale? (MAU, catalog size, QPS)
   - Latency SLA? (< 200ms? < 50ms?)

2. DATA (3-5 min)
   - What training data is available?
   - How are labels derived?
   - Feature store needed?
   - Data freshness requirements?

3. MODELING (5-7 min)
   - Two-stage vs. single-stage?
   - Candidate generation approach?
   - Ranking model architecture?
   - Cold-start strategy?

4. EVALUATION (3-5 min)
   - Offline metrics?
   - Online A/B testing plan?
   - What does "better" mean?

5. SERVING (3-5 min)
   - Latency requirements → ANN search, request batching, caching?
   - Feature computation online vs. offline?
   - Model update frequency?

6. MONITORING (2-3 min)
   - Feature drift?
   - Prediction distribution shift?
   - Business metric dashboards?

Sample System Design: LLM-Powered Support Chat

Problem: Design a customer support chat that uses an LLM to answer questions about a software product.

Architecture decision tree:

1. Pure LLM prompting → high quality but expensive, no private knowledge
2. RAG + LLM → answers from private docs, cost-controlled, my recommendation
3. Fine-tuned smaller LLM → only if volume is high (>1M requests/day) and quality of RAG is insufficient

Chosen architecture (RAG + gpt-4o-mini):

Ticket submitted → Query embedding (bge-small)
                → Hybrid search (pgvector + BM25)
                → Top-5 chunks reranked (cross-encoder)
                → Prompt assembly
                → GPT-4o-mini (structured output: answer + sources + confidence)
                → Confidence < 0.7 → escalate to human agent

Serving: FastAPI + Redis semantic cache (60% expected hit rate on FAQ questions) → reduces LLM calls by 60%, cost by ~60%.

Monitoring: response confidence distribution, citation accuracy (LLM judge sample), escalation rate, CSAT scores linked to session IDs.

Tradeoffs: gpt-4o-mini may misunderstand complex technical questions → mitigated by confidence threshold + escalation. pgvector may miss exact API names → mitigated by BM25 hybrid retrieval.

The Agents and Evals Round

Most AI engineering loops now include a round, or a large chunk of the system design round, on agents. It usually starts from a scenario ("design an agent that resolves tier-1 support tickets") and probes four things in order.

1. Tool and permission design. What tools does the agent get, and what can it not do? Interviewers want to hear tools scoped per task, side effects gated behind approvals, and a clear statement of the worst case. If you describe the agent before you describe its boundaries, you have already lost points.
2. The harness. How is a proposed tool call validated, executed, logged, and fed back? What happens when a tool fails? What stops an infinite loop? Draw the loop with the harness as a box around the model, and name the budget, verification, and tracing layers.
3. Evaluation. How do you know it works, and how do you know it still works after you change the prompt? Describe a task suite built from real tickets, trajectory grading (right tools, right order, no loops), a validated judge for the free-text parts, and a CI gate. Have a number ready from a project: "our suite has 60 tasks; the last prompt change moved trajectory pass rate from 81% to 88% and cost per ticket down 30%."
4. Cost and model choice. Which requests need a reasoning model, which go to a small model, and how do you decide? Mention routing and prompt caching. Interviewers at product companies care about this because it is where the budget goes.

If you have built the deliverables in the agent-engineering modules of this path (an MCP server, a harness with permissions and traces, an agent eval suite, a red-team report), you can answer all four from experience, and that is the difference between a candidate who read about agents and one who ships them.

Behavioral Questions: The STAR Framework

Every behavioral answer needs:

• Situation: brief context (1-2 sentences)
• Task: what you specifically needed to accomplish
• Action: what you did - emphasize your decisions and reasoning
• Result: quantified outcome + what you learned

Q: "Tell me about a time an ML model you built didn't work as expected."

S: "We deployed a churn prediction model for our B2B product that showed 87% AUC in offline eval."
T: "After launch, the sales team reported the model was flagging wrong accounts."
A: "I pulled production predictions and compared feature distributions - the model was trained on a cohort of churned customers from 2022, but the product had added an enterprise tier in 2023 that had very different usage patterns. I rebuilt the training pipeline with a rolling 6-month window and added feature distribution monitoring."
R: "The new model improved precision on enterprise accounts by 34%. More importantly, I added daily PSI monitoring so future distribution shifts would alert us before they affect production."

Prepare 5-6 STAR stories covering: a model failure, a data quality issue, a design tradeoff you made, a time you pushed back on a stakeholder, and a successful cross-team collaboration.

Common Mistakes and Bad Instincts

Jumping to architecture before clarifying constraints. Every system design answer that opens with "I would use a Transformer-based model" without clarifying scale, latency, and data availability looks unprepared. Spend the first 2-3 minutes asking questions - interviewers expect and reward this.

Overclaiming on offline metrics. An interviewer will ask "how do you know the 92% AUC is good?" If your answer is "it's higher than the baseline," that is weak. Know what the human-level performance is, what the business-relevant operating point is, and how offline metrics correlate with online metrics for your task.

Not discussing what did not work. Candidates who present only successes seem either inexperienced or not forthcoming. Interviewers find candidates more credible when they honestly describe failures and what they learned.

Where to Go Next

• capstone-build-and-operate-a-production-style-ai-system: the capstone gives you a full, end-to-end project to anchor every interview story
• portfolio-conversion-turning-engineering-work-into-ml-evidence: ensure every interview answer is backed by portfolio evidence