What Makes Data Useful for ML?

Not All Data Trains Good Models

A common misconception is that any sufficiently large dataset can train a good ML model. In reality, data needs several specific properties to be useful for supervised learning. Understanding these properties lets you evaluate AI project proposals, identify risks before starting, and communicate data requirements to ML teams.

Property 1: A Clear, Measurable Label

Every supervised ML model learns to predict something. That something must be:

Clearly defined: "Customer will be satisfied" is not a clear label. "Customer gives a 4 or 5 star rating within 7 days of resolving their support ticket" is clear.

Observable: You need to know the correct answer for historical examples. A model predicting fraud needs labeled historical examples of fraudulent and legitimate transactions. If you cannot know whether historical transactions were fraudulent, you cannot train the model.

Consistent: Labels must be produced by the same process across all examples. If some labels came from one team's judgment and others from a different team's judgment, the model learns inconsistency as much as it learns the signal.

Ambiguous labels are the single most common root cause of disappointing ML project outcomes.

Property 2: Enough Examples

More labeled examples generally produce better models. How many is enough depends on the complexity of the problem:

• Simple patterns (is this email spam?): thousands of examples may suffice
• Complex patterns (what medical treatment is best for this patient?): millions of examples may not be enough

A practical rule: if your dataset has fewer than a few thousand labeled examples for each category you care about, supervised learning is risky. Consider whether more data can be collected before investing in model building.

Volume also matters across categories. If 99% of your examples are "not fraud" and 1% are "fraud," a model that always predicts "not fraud" achieves 99% accuracy while being completely useless. Severe class imbalance requires special handling.

Property 3: Relevant Features

The model learns from the columns (features) available in the dataset. If the information needed to make the prediction is not in the data, no algorithm can find it.

A useful question: "If I were a very smart person who could only see the columns in this dataset, could I do better than random guessing?" If the answer is no, the model will not either.

Common feature problems:

• Irrelevant features: Columns that have no causal or correlational relationship with the label add noise
• Leaky features: Features that are derived from or caused by the label (using future information to predict the past)
• Missing important features: The key predictor is not in the dataset

Property 4: Representative Coverage

The dataset must cover the range of situations the model will encounter in deployment. If the training data came from one region, one time period, or one customer segment, the model will only work well for situations similar to those in the data.

Ask: who or what is not represented in this data? The gaps in coverage become the gaps in model performance.

Property 5: Data That Reflects the Deployment Context

The data used to train the model must be generated by a process similar to the one the model will operate in. If you train a model on manually collected clinical data but deploy it on data from an automated sensor system, the distributions will differ and the model may fail.

This is called distribution shift, and it is the most common cause of models that perform well in testing but fail in production.

The Data Assessment Template

When evaluating whether a dataset is suitable for an ML project:

Question	Why It Matters
What is the label? Is it clearly defined and consistently applied?	Ambiguous labels produce unreliable models
How many labeled examples exist? How balanced are the classes?	Too few examples or severe imbalance limits model quality
What features are available? Do they contain the signal needed?	Missing key features limits what any model can learn
What population does this data represent? Who is missing?	Coverage gaps become performance gaps
How was the data collected? What biases might that introduce?	Collection process determines systematic errors
How recent is the data? Does the distribution change over time?	Stale data produces models that drift from current reality

What This Means in Practice

When someone proposes an ML project, these questions - asked early - prevent the most expensive mistakes. A project that answers all six questions satisfactorily is in a good position to proceed. A project with unclear labels and uncertain feature availability is a high-risk investment regardless of model sophistication.

You do not need to be a data scientist to ask these questions. They are data quality questions, and data quality is everyone's concern.

Where to Go Next

The final module of Phase 3 covers visualization - the skill that turns the exploration you have learned to do in Python into communicable insights.