Unsupervised Learning and Recommendation Basics

When There Are No Labels

Supervised learning requires labeled data - examples paired with the correct answer. But most data in the world has no labels. User behavior logs, product catalogs, documents, images: these come in bulk with no pre-attached "correct" interpretation.

Unsupervised learning finds structure in data without labels. Recommendation systems are one of the highest-impact applications. Understanding both makes you useful in a much wider range of ML problems.

Clustering: Grouping Without Answers

Clustering assigns data points to groups based on similarity. The key challenge: you do not know ahead of time what the groups should be, or even how many there are.

K-Means: Simple and Powerful

K-means divides data into k clusters by iteratively assigning points to the nearest cluster center and recomputing the centers.

pythonCopy
from sklearn.cluster import KMeans
import numpy as np

kmeans = KMeans(n_clusters=5, random_state=42, n_init=10)
kmeans.fit(X)
labels = kmeans.labels_        # cluster assignment for each point
centers = kmeans.cluster_centers_  # center of each cluster

When to use it: When you have a reasonable guess for the number of clusters and your data is approximately spherical (similar size and density clusters).

When it fails: Clusters of very different sizes, non-spherical clusters (rings, crescents), or data with noise. K-means also requires you to choose k - use the elbow method or silhouette score to guide this.

DBSCAN: Density-Based Clustering

DBSCAN groups points that are densely packed together, marking sparse points as outliers. It does not require you to specify the number of clusters.

pythonCopy
from sklearn.cluster import DBSCAN

dbscan = DBSCAN(eps=0.5, min_samples=10)
labels = dbscan.fit_predict(X)
# label of -1 means the point was marked as noise/outlier

When to use it: Anomaly detection, geographic data, clusters of irregular shape, when the number of clusters is unknown.

Dimensionality Reduction: Seeing Structure in High Dimensions

Real datasets often have hundreds or thousands of features. Dimensionality reduction compresses them into a smaller number of dimensions that capture most of the structure - for visualization, for removing noise, or for creating better features.

PCA: Linear Compression

Principal Component Analysis finds the directions (principal components) of maximum variance in the data and projects data onto them.

pythonCopy
from sklearn.decomposition import PCA

pca = PCA(n_components=50)
X_reduced = pca.fit_transform(X)
print(f"Variance explained: {pca.explained_variance_ratio_.sum():.2%}")

The first principal component is the direction where data varies most. The second is the direction of second-most variance, orthogonal to the first. Together, the top n components capture n axes of the most important variation.

Use cases: Noise reduction before training, visualization (project to 2D), removing correlated features, speeding up training.

UMAP and t-SNE: Nonlinear Visualization

PCA is linear - it cannot capture curved or folded structure in data. UMAP and t-SNE are nonlinear methods used primarily for 2D or 3D visualization of high-dimensional data (e.g., visualizing word embeddings or image feature spaces).

pythonCopy
import umap

reducer = umap.UMAP(n_components=2, random_state=42)
embedding = reducer.fit_transform(X)

Important limitation: UMAP and t-SNE are for visualization, not for creating features for downstream models. The axes are not interpretable and distances between clusters are not meaningful across runs.

Embeddings: Dense Representations of Meaning

An embedding is a dense, low-dimensional vector representation of a high-dimensional or discrete object. A product ID, a word, a user - all can be represented as a vector of floats that captures semantic meaning.

The key property of a good embedding: similar things should be close together in the embedding space. If word2vec represents "king" and "queen" as vectors that are close together, that is a useful representation.

Why Embeddings Matter

Embeddings are the foundation of modern ML:

• Word embeddings (word2vec, GloVe) → modern NLP
• Item embeddings → recommendation systems
• Image embeddings → visual similarity search
• User embeddings → personalization

They compress discrete objects into continuous spaces where arithmetic operations make semantic sense: king - man + woman ≈ queen in word2vec is the classic example.

Recommendation Systems: Unsupervised Learning in Production

Recommendation systems sit behind feeds, search results, playlists, shopping pages, and video platforms. The mechanics matter because small ranking changes can shift what millions of users see.

Collaborative Filtering

Collaborative filtering recommends items based on the behavior of similar users. It does not need to understand what the items are - only that users with similar history tend to like similar things.

User-based CF: Find users similar to the target user; recommend what they liked.

Item-based CF: Find items similar to what the target user has liked; recommend those.

Matrix factorization: Decompose the user-item interaction matrix into user embeddings and item embeddings. Dot products between user and item embeddings predict ratings.

R ≈ U × V^T

Where:
R = user-item interaction matrix (ratings, clicks, purchases)
U = user embedding matrix (n_users × k)
V = item embedding matrix (n_items × k)
k = embedding dimension (latent factors)

Matrix factorization is the core idea behind SVD-based recommenders and modern two-tower neural recommendation systems.

Content-Based Filtering

Content-based filtering recommends items similar to items the user has previously engaged with, based on item attributes or features.

Example: A user who watched three science fiction films gets recommended more science fiction films based on genre, director, and actor embeddings.

Advantage over CF: Works for new items (no cold-start problem for new items). Does not need other users' data.

Disadvantage: Narrow recommendations - tends to recommend more of the same, not discovering new tastes.

Hybrid Systems

Production recommendation systems almost always combine multiple approaches:

• Candidate generation: collaborative filtering to generate thousands of candidates
• Ranking: a supervised model that scores candidates using user features, item features, context, and interaction history
• Post-processing: business rules, diversity injection, freshness boosting

This two-stage architecture separates the recall problem (find relevant candidates) from the ranking problem (order them correctly), allowing each stage to be optimized independently.

Evaluating Unsupervised Models

Evaluating clustering without labels is inherently hard. Common approaches:

Silhouette score: Measures how similar a point is to its own cluster versus neighboring clusters. Ranges from -1 to 1; higher is better.

Domain evaluation: Show cluster samples to domain experts. Do the clusters correspond to meaningful categories?

Downstream task performance: If you cluster users for targeting, do campaigns sent to clusters outperform random targeting? This is the ultimate test.

For recommendation systems, offline metrics include:

• Precision@K: Of the top K recommendations, how many did the user engage with?
• Recall@K: Of all items the user engaged with, how many were in the top K recommendations?
• NDCG@K: Normalized discounted cumulative gain - rewards getting relevant items higher in the ranking.

Online evaluation (A/B testing) is required before deployment. Offline metrics do not reliably predict online performance.

Common Mistakes and Bad Instincts

Choosing k arbitrarily in k-means. Use the elbow method (plot inertia vs. k, look for the knee) or silhouette score to guide the choice. Do not just pick k=5 because it feels right.

Treating UMAP clusters as ground truth. UMAP visualizations can create visually compelling clusters that do not correspond to meaningful structure. Validate with domain knowledge.

Evaluating recommendations offline and calling it done. Offline metrics (precision@K, NDCG) correlate poorly with online performance. Unexpectedly popular items, diversity, and novelty effects only show up in live traffic.

Forgetting the cold-start problem. Collaborative filtering cannot recommend to new users or new items with no interaction history. Plan for this from the beginning.

Where to Go Next

Unsupervised learning and embeddings are covered in Module 15 (Unsupervised Learning, Representation, and Dimensionality Reduction) of the College Student path. Recommendation systems appear as a domain case study in Phase 4. The SWE path covers similar terrain in Module 12 (Representation Learning, Embeddings, and Similarity) and Module 13 (domain modeling track options).

When There Is No Label

Unsupervised learning is useful when you do not have a clean target but still need structure. You may want to group users, compress text, detect anomalies, or find similar items.

The danger is that unsupervised methods always produce output. K-means will always create clusters. PCA will always produce components. A nearest-neighbor search will always return something. The question is whether the structure means anything.

Clustering: Useful Only With Interpretation

Imagine clustering customers based on product behavior. You get five clusters. That is not the result. The result is what those clusters mean:

• Trial users exploring lightly
• Power users with deep adoption
• Dormant accounts
• Support-heavy accounts
• Seasonal users

To make clusters useful, profile them:

• Size of each cluster
• Key feature differences
• Retention or revenue behavior
• Representative examples
• Stability over time
• Actionability for a team

If no team can take a different action because a cluster exists, the cluster is probably just decoration.

Dimensionality Reduction

Dimensionality reduction turns many features into fewer features while preserving useful structure. PCA is the classic method. Embeddings are the modern workhorse.

Use dimensionality reduction for:

• Visualization
• Noise reduction
• Similarity search
• Feature compression
• Understanding dominant directions of variation

Do not use a 2D plot as proof that groups are real. Visualizations are diagnostic tools, not statistical guarantees.

Recommendation Systems in Plain Terms

Most recommendation systems answer a ranking question: for this user, which items should appear first?

Common signals include:

• User-item interactions: clicks, purchases, ratings, watch time
• Item content: text, category, metadata, image embeddings
• User context: location, time, device, session history
• Popularity and freshness

Early systems can be simple:

• Popular items by category
• Similar items based on metadata
• Users who liked this also liked that

More mature systems often use two stages:

1. Candidate generation: quickly find a few hundred plausible items
2. Ranking: sort those items using richer features and a stronger model

Offline Metrics Are Not Enough

Recommendation quality depends on product behavior. A model can optimize clicks while hurting satisfaction. It can over-personalize and trap users in repetitive content. It can amplify popularity and bury new items.

Track:

• Click-through rate
• Conversion
• Long-term retention
• Diversity
• Freshness
• Coverage across items
• User complaints or hides

The best recommender is not always the one with the highest immediate engagement. It is the one that supports the product's long-term value.

Evaluation Without Labels

When labels are weak or absent, combine evidence:

• Human inspection
• Stability checks
• Downstream task performance
• A/B tests
• Cluster/action usefulness
• Retrieval relevance judgments

Unsupervised learning becomes powerful when you connect discovered structure to decisions.

Quick Self-Assessment

You understand this topic when you can explain the main tradeoff, name the most likely failure mode, and describe how you would test the work before trusting it. Do that in writing. Short written explanations expose vague thinking quickly.

Final Rule

Unsupervised learning is strongest when it helps people inspect, organize, retrieve, or prioritize. Treat the output as a proposed structure, then validate it against human judgment and downstream decisions.

Similarity Search Beyond Recommendations

Similarity search also powers duplicate detection, semantic search, support ticket routing, code search, image search, and document clustering. The same principle appears repeatedly: represent items as vectors, then retrieve nearby vectors.

Quality depends on representation. A generic embedding may work for broad semantic similarity but fail on domain-specific terms, product names, legal language, medical abbreviations, or code identifiers. Always evaluate embeddings on examples from your actual use case.

Anomaly Detection

Anomaly detection is another common unsupervised pattern. It tries to identify examples that do not look like the normal population: unusual transactions, failing machines, suspicious logins, broken sensors, or unexpected traffic.

The key challenge is that rare does not always mean bad. A VIP customer, a holiday spike, or a new product launch may look anomalous but be completely valid. Treat anomaly detection as a triage system. It should surface cases for inspection, not automatically declare guilt.

Evaluate anomaly systems with reviewer feedback, incident discovery rate, false alarm burden, and time saved. The human workflow matters as much as the model.