ML Glossary: Terms Every Practitioner Should Know

Use this as a working reference while reading the ML Foundations and MLOps paths. Each term is intentionally practical: what it means, when it matters, and what mistake it helps you avoid.

How to Use This Glossary

• If you are debugging model quality, start with Confusion Matrix, Precision, Recall, F1, AUC-ROC, AUC-PR, Overfitting, Underfitting, and Validation Set.
• If you are building production systems, start with Feature Store, Inference, Latency, Monitoring, Training-Serving Skew, and Data Drift.
• If you are working with LLMs, start with Embedding, Tokenization, Vector Database, RAG, and Prompt Engineering.

This glossary favors practical clarity over mathematical completeness. Each term includes what it means in context and when you would use it.

A/B Test

An experiment that randomly splits users into two groups to compare two interventions. The control group gets the current behavior; the treatment group gets the new one. Results are analyzed statistically to determine if the difference is real. Useful for validating that offline model improvements translate to business outcomes.

AUC-ROC (Area Under the Receiver Operating Characteristic Curve)

Measures how well a binary classifier separates positive and negative classes across all thresholds. A value of 1.0 is perfect; 0.5 is random. Useful for comparing models when you have not yet committed to a threshold. For severely imbalanced data (fraud, rare disease), prefer AUC-PR instead.

AUC-PR (Area Under the Precision-Recall Curve)

Measures classifier performance specifically on the positive class. More informative than AUC-ROC when positive examples are rare. A model with 99.9% AUC-ROC on 0.1% fraud data may still have terrible AUC-PR - it might correctly classify all negatives but miss all positives.

Batch Normalization

A technique that normalizes layer activations during training, reducing internal covariate shift. Applied after a linear layer and before the activation function. Enables higher learning rates, acts as regularization, and makes training more stable. Standard in CNNs; less universal in transformers, which use Layer Normalization.

Cold Start

The problem of making predictions for new users or new items with no historical interaction data. Common in recommendation systems. Solutions: content-based features (what does the item look like?), onboarding flows (ask users about preferences), and exploration strategies (serve new items to a fraction of users to gather data quickly).

Confusion Matrix

A 2×2 table for binary classification: True Positives, True Negatives, False Positives, False Negatives. From these four numbers, you can compute accuracy, precision, recall, F1, and specificity. Always look at the confusion matrix, not just aggregate metrics, when evaluating a classifier.

Cross-Entropy Loss

The standard loss function for classification. For binary classification: -[y·log(p) + (1-y)·log(1-p)]. For multi-class: -Σ y_i·log(p_i). Penalizes confident wrong predictions much more than uncertain wrong predictions, which drives the model to be well-calibrated.

Data Leakage

When information from the future or from the target variable leaks into the training features, inflating offline metrics. The model appears accurate but fails in production because the leaked feature is not available at prediction time. Common sources: computing features using data after the event timestamp, including derived features that encode the label.

Dropout

A regularization technique where a random fraction of neurons are set to zero during training. This prevents neurons from co-adapting too closely and forces more robust feature representations. Applied during training only - at inference, all neurons are active and outputs are scaled by the keep probability.

Early Stopping

Stopping training when validation loss stops improving, instead of training for a fixed number of epochs. Prevents overfitting without requiring a penalty term. Requires a separate validation set. Use with a "patience" parameter (e.g., stop if val loss does not improve for 10 consecutive epochs).

Embedding

A dense vector representation of a discrete entity (word, user, product, category). Learned from data such that similar entities are close in vector space. Embeddings reduce dimensionality compared to one-hot encoding and allow the model to generalize. The embedding layer is usually the first layer in models that handle categorical inputs.

F1 Score

The harmonic mean of precision and recall: 2 · precision · recall / (precision + recall). Use when both false positives and false negatives matter and you cannot choose between them. F-beta generalizes this: beta > 1 weights recall more, beta < 1 weights precision more.

Feature Importance

A measure of how much each feature contributes to model predictions. For tree models (Random Forest, XGBoost): typically measures the reduction in impurity from splits on that feature. For linear models: the magnitude of the coefficient. For neural networks: usually computed via SHAP or permutation importance. Useful for debugging, model compression, and explaining predictions to stakeholders.

Fine-Tuning

Adapting a pretrained model to a new task by continuing training on task-specific data. The pretrained weights serve as initialization. The key hyperparameter is the learning rate - too high and you destroy the pretrained representations; too low and learning is slow. Typically 10–100x smaller than the learning rate used for pretraining.

Gradient Vanishing / Exploding

Problems in training deep networks where gradients become extremely small (vanishing) or large (exploding) during backpropagation, making learning unstable. Solutions: residual connections (skip connections), batch normalization, gradient clipping, careful weight initialization (He, Xavier).

Hyperparameter

A parameter that controls the learning process and is set before training, not learned from data. Examples: learning rate, number of hidden layers, regularization strength, batch size. Tuning hyperparameters is a meta-optimization problem - it is done outside the training loop using cross-validation or a held-out validation set.

NDCG (Normalized Discounted Cumulative Gain)

A ranking metric that measures the quality of a ranked list, with higher positions weighted more. A perfect ranking (most relevant items first) scores 1.0. Used for search and recommendation systems. NDCG@K evaluates only the top K results.

Overfitting

When a model learns the training data too well, including its noise, and fails to generalize to new examples. Symptoms: very low training loss, high validation loss. Solutions: more data, regularization (L1/L2, dropout), simpler model, early stopping, data augmentation.

Precision

Of all the examples predicted positive, what fraction are actually positive? TP / (TP + FP). High precision means few false alarms. Critical when false positives are costly (blocking legitimate transactions, spam-flagging real emails).

Recall (Sensitivity, True Positive Rate)

Of all the actual positive examples, what fraction did the model find? TP / (TP + FN). High recall means few misses. Critical when false negatives are costly (missed fraud, missed cancer detection).

Regularization

Techniques that reduce model complexity to improve generalization. L2 (Ridge): penalizes large weights, drives them toward zero. L1 (Lasso): drives some weights to exactly zero, producing sparse models. Dropout: stochastically removes neurons during training. Data augmentation: implicit regularization by increasing effective training set size.

Tokenization

Converting text into tokens - the atomic units a language model processes. Common approaches: word tokenization (splits on spaces, loses morphology), character tokenization (maximally granular), subword tokenization (BPE, WordPiece - the standard in modern LLMs). Subword tokenization handles unknown words by splitting them into known subword units.

Training-Serving Skew

When the feature values used during training differ from those used at inference time. Causes: different code paths for training vs. serving, different data sources, different preprocessing logic. One of the most common silent failure modes in production ML.

Underfitting

When a model is too simple to capture the patterns in the data. Symptoms: high training loss and high validation loss. Solutions: more complex model, better features, more training data, less regularization.

Validation Set

A held-out subset of data used to tune hyperparameters and make modeling decisions during development. Distinct from the test set, which should be used only once to report final performance. If you use the test set to make decisions, you are effectively training on it and your reported metrics will be optimistic.

Agent Engineering and Post-Training Terms

Added September 2026. These are the terms that show up in agent, harness, and post-training conversations and that did not exist, or did not matter, when the original glossary was written.

A2A (Agent-to-Agent Protocol)

A protocol for one agent to discover another (via a published "agent card"), delegate a task to it, and receive results. Complements MCP: MCP connects an agent to tools, A2A connects agents to each other. Most systems do not need it until they have more than one independently owned agent.

Agent Harness

The deterministic code around a model that turns it into an agent: the tool list, the permission and budget checks, the verification steps, the memory, and the trace log. The formula "agent = model + harness" is the standard way to say that reliability lives in the harness, not the model.

Checkpointing

Saving an agent's full run state (current step, tool results so far, pending approvals) after every step so the run can resume from the last good step after a crash, instead of restarting. The foundation of durable execution.

Compaction

Summarizing or dropping earlier context when the window fills. Always lossy; the engineering is in deciding what to keep, keeping raw artifacts retrievable by ID, and measuring how often it happens.

Computer-Use Agent

An agent that operates a graphical interface the way a person does: it looks at a screenshot, decides where to click or type, and acts. Contrast with agents that call APIs or run shell commands. Slower and more fragile, but works with software that has no API.

Context Engineering

The discipline of designing everything the model sees on a given turn: system prompt, tool definitions, retrieved documents, memory, and prior turns, under a fixed token budget. Succeeded "prompt engineering" as the senior skill once tool lists and retrieval made the prompt the smaller part of the context.

DPO (Direct Preference Optimization)

A post-training method that trains a model directly on pairs of (preferred, rejected) responses without a separate reward model. Simpler and cheaper than RLHF with PPO; the standard first step when you have preference data.

Durable Execution

Running a multi-step workflow so that it survives process crashes, restarts, and long waits (for a human, for an external system) without losing progress. Implemented with checkpointing and a workflow engine.

Excessive Agency

The failure where an agent has more permissions or tools than its task requires, so that a mistake or an injection can cause disproportionate harm. Named in the OWASP Top 10 for Agentic Applications; the fix is scoping tools per task.

GRPO (Group Relative Policy Optimization)

A reinforcement learning algorithm for post-training that samples a group of responses per prompt and uses each response's reward relative to the group's mean as its advantage. Drops the value model that PPO needs, which makes it cheaper and more stable. The workhorse behind most reasoning models.

Guardian Pattern

Placing a second, narrowly scoped model or rule set between the agent and its tools (or between untrusted content and the agent) to inspect and block dangerous actions or inputs. A harness-layer defense, not a replacement for permissions.

Human-in-the-Loop (HITL)

Designing the workflow so that a person approves specific actions (usually irreversible ones) before they execute. In a durable agent, the approval is a state the run waits in, and the approval is recorded in the trace.

LLM-as-Judge

Using a model to grade another model's output against a rubric. Cheap and scalable; must itself be validated against human labels on a sample, because judges have systematic biases (length, position, self-preference).

MCP (Model Context Protocol)

The open standard for connecting a model to tools, data, and prompts. An MCP server exposes tools (callable functions), resources (readable data), and prompts (reusable templates); an MCP client (an agent, an IDE, a chat app) discovers and calls them. Supported by every major provider; the default way to package integrations for agents.

Memory Poisoning

An attack or accident where bad content enters an agent's long-term memory and then influences every future run. The reason long-term memory needs a write policy, provenance, and the ability to inspect and delete entries.

Model Routing

Sending each request to the cheapest model that can handle it, based on rules, a classifier, or a first attempt with a small model plus a verifier. A cascade is routing where escalation happens after a failed attempt.

Prompt Caching

Reusing the computed state (the KV cache) for a prompt prefix across requests. Providers bill cached tokens at a discount; serving frameworks do it automatically. Requires stable content to come first in the prompt. Cache hit rate is an operational metric.

Prompt Injection

Content the model reads (a web page, an email, a document, a tool result) that contains instructions the model then follows. Structurally unsolved because the model sees instructions and content as one sequence. Contained, not prevented, by limiting what the agent can do.

Reasoning Model

A model post-trained (usually with RLVR) to spend tokens on intermediate reasoning before answering. Better on verifiable, multi-step tasks; slower and more expensive per request. Many expose a thinking budget you can set.

RLVR (Reinforcement Learning with Verifiable Rewards)

Post-training where the reward is computed by a checker (unit tests pass, the numeric answer matches, the output parses) instead of a learned reward model. The dominant recipe for reasoning models because it scales without human labels. Only works where you can write the verifier.

Sandbox

An isolated execution environment (container, VM, restricted interpreter) where an agent's code or tool calls run so that a mistake or an injection cannot reach the host system, the network, or production data.

SLM (Small Language Model)

A model in roughly the 1B to 8B parameter range that runs on a laptop, a phone, or a single small GPU. Competitive with older large models on narrow tasks; the standard choice for high-volume routing, classification, and extraction paths.

Speculative Decoding

A serving technique where a small draft model proposes several tokens and the large model verifies them in one pass, accepting the matching prefix. Same output distribution, two to three times more tokens per expensive forward pass.

Test-Time Compute

Spending more inference compute on a single request to get a better answer: longer reasoning, sampling several answers and voting, or searching over candidate solutions. A dial that trades latency and cost for accuracy, with diminishing returns.

Trajectory

The full sequence of an agent's reasoning, tool calls, tool results, and intermediate outputs across a run. Agent evaluation grades the trajectory (did it call the right tools, in a sensible order, without looping) and not only the final answer.

Verifiable Reward

A reward you can compute mechanically: tests pass, the answer matches, the schema validates. The precondition for RLVR and the cheapest kind of eval you can build.