ML Notes
HubAI vs ML vs Deep Learning — Machine LearningApplications of Machine LearningHistory of Machine LearningMachine Learning RoadmapWhat is Machine Learning?Calculus for Machine LearningData Preprocessing BasicsLinear Algebra for Machine LearningProbability Basics for Machine LearningPython for Machine LearningStatistics Basics for Machine LearningBias vs Variance TradeoffModel Evaluation OverviewOverfitting and UnderfittingReinforcement Learning Fundamentals — Machine LearningSupervised LearningTraining vs Testing DataTypes of Machine LearningUnsupervised LearningData Cleaning — Machine LearningData Collection for Machine LearningData Visualization for MLEncoding Categorical DataFeature EngineeringFeature ScalingHandling Missing ValuesTrain-Test SplitElastic Net RegressionLasso Regression (L1 Regularization)Multiple Linear RegressionPolynomial RegressionIntroduction to RegressionRegression ProjectsRidge Regression (L2 Regularization)Simple Linear RegressionIntroduction to ClassificationClassification ProjectsDecision TreesK-Nearest Neighbors (KNN)Logistic RegressionNaive Bayes ClassifierRandom Forest ClassificationSupport Vector Machine (SVM)XGBoost for ClassificationClustering in Machine Learning: Complete Guide to Unsupervised LearningClustering Projects: Real-World Applications and Implementation GuidesDBSCAN ClusteringGaussian Mixture Models (GMM): Probabilistic Clustering for Advanced Data AnalysisHierarchical ClusteringK-Means ClusteringDimensionality Reduction: Comprehensive Guide to Feature Reduction TechniquesFeature Selection Techniques: Choosing Informative Variables for ML ModelsLinear Discriminant Analysis (LDA): Supervised Dimensionality ReductionPrincipal Component Analysis (PCA)t-SNE: Non-Linear Dimensionality Reduction for Data VisualizationUMAP: Uniform Manifold Approximation and Projection for Modern Data VisualizationAdaBoost: Adaptive Boosting Algorithm Explained with ExamplesBagging: Bootstrap Aggregating for Reducing Model VarianceBoosting: Sequential Ensemble Learning for Bias and Variance ReductionEnsemble Learning: Complete Guide to Combining ML Models for Better PredictionsGradient Boosting: Building Powerful ML Models Through Sequential OptimizationRandom Forest: Complete Guide to Tree Ensembles and Feature ImportanceStacking: Advanced Ensemble Method Using Meta-Learners for Superior ML PredictionsActivation Functions in Neural NetworksBackpropagation: Complete Algorithm for Training Neural NetworksIntroduction to Deep Learning: Neural Networks, Architectures, and ApplicationsDeep Learning Projects: Hands-On Applications and Case StudiesNeural Networks Basics: Perceptron, Architecture, and TrainingPerceptron: Foundational Single-Neuron Classifier for Binary ClassificationPyTorch Basics: Building and Training Neural NetworksTensorFlow Basics: Building and Training Neural NetworksIntroduction to Natural Language Processing: Fundamentals and ApplicationsNLP Projects: Practical Applications from Sentiment Analysis to Machine TranslationStemming and Lemmatization: Reducing Words to Root Forms in NLPText Preprocessing: Essential Steps for NLP ApplicationsTF-IDF: Term Frequency-Inverse Document Frequency for Text Feature ExtractionTokenization in NLP: Breaking Text into Meaningful UnitsTransformers: The Architecture Behind Modern NLP ModelsWord Embeddings: Representing Words as Vectors for NLPIntroduction to Computer VisionComputer Vision ProjectsImage Classification: Building and Deploying CNN ModelsImage Processing BasicsImage Segmentation — Machine LearningObject DetectionOpenCV BasicsAgents and Environments in RLDeep Q-Networks (DQN)Policy Gradient Methods — Machine LearningQ-Learning AlgorithmReinforcement Learning: IntroductionReinforcement Learning ProjectsAccuracy, Precision, Recall: Classification Metrics Explained for ML InterviewConfusion Matrix: Complete Guide to Classification EvaluationCross-Validation: Comprehensive Guide to Robust Model EvaluationF1 Score: Balanced Classification Metric for Imbalanced DataGrid Search: Complete Guide to Hyperparameter Tuning in Machine LearningHyperparameter Tuning: Complete Guide to Optimizing Machine Learning ModelsModel Selection: Choosing the Best Algorithm for Your ML ProblemROC and AUC: Complete Guide to Classification Threshold EvaluationBuild an AI Assistant with LLMsBuild a Conversational ChatbotCustomer Churn PredictionFace Recognition System — Machine LearningHouse Price PredictionRecommendation System ProjectSentiment Analysis Project — Machine LearningSpam Email ClassifierML Case Study Questions for InterviewsML Coding Questions for Technical InterviewsMachine Learning Interview CheatsheetEssential Machine Learning Interview QuestionsStatistics for Machine Learning InterviewsDatasets for ML PracticeImportant ML FormulasMachine Learning GlossaryRecommended ML Books and CoursesCI/CD for Machine LearningDocker for Machine LearningFastAPI for Machine Learning APIsFlask API for Machine LearningMLOps: IntroductionModel Deployment StrategiesML Model Monitoring & Observability
Comprehensive guide to unsupervised learning including clustering, dimensionality reduction, anomaly detection, and association rules with Python examples.
Unsupervised learning is the art of finding hidden patterns in data without any labels or guidance. The algorithm must discover structure on its own — like organizing a messy closet without anyone telling you the categories.
Core Concept
Example
Supervised: X ──── y (known) → Learn mapping X → y
Unsupervised: X ──── ? (unknown) → Discover structure in X
Types of Unsupervised Learning
Unsupervised Learning
├── Clustering
│ ├── Partitional (K-Means, K-Medoids)
│ ├── Density-based (DBSCAN, OPTICS)
│ ├── Hierarchical (Agglomerative, Divisive)
│ └── Model-based (Gaussian Mixture Models)
│
├── Dimensionality Reduction
│ ├── Linear (PCA, SVD)
│ └── Non-linear (t-SNE, UMAP, Autoencoders)
│
├── Anomaly Detection
│ ├── Isolation Forest
│ ├── Local Outlier Factor
│ └── One-Class SVM
│
└── Association Rules
└── Apriori, FP-Growth
Clustering: Grouping Similar Data
python example
from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.datasets import make_blobs, make_moons
from sklearn.metrics import silhouette_score
import numpy as np
# Create sample data
X_blobs, _ = make_blobs(n_samples=300, centers=4, random_state=42)
# K-Means Clustering
kmeans = KMeans(n_clusters=4, random_state=42, n_init=10)
kmeans_labels = kmeans.fit_predict(X_blobs)
kmeans_score = silhouette_score(X_blobs, kmeans_labels)
# DBSCAN Clustering
dbscan = DBSCAN(eps=0.8, min_samples=5)
dbscan_labels = dbscan.fit_predict(X_blobs)
dbscan_score = silhouette_score(X_blobs, dbscan_labels)
# Hierarchical Clustering
hier = AgglomerativeClustering(n_clusters=4)
hier_labels = hier.fit_predict(X_blobs)
hier_score = silhouette_score(X_blobs, hier_labels)
print("Clustering Comparison (Silhouette Score - higher is better):")
print(f" K-Means: {kmeans_score:.4f}")
print(f" DBSCAN: {dbscan_score:.4f}")
print(f" Hierarchical: {hier_score:.4f}")Dimensionality Reduction
python example
from sklearn.decomposition import PCA
from sklearn.datasets import load_digits
import numpy as np
# Load high-dimensional data (64 features)
digits = load_digits()
X = digits.data
print(f"Original shape: {X.shape}") # (1797, 64)
# Reduce to 2D for visualization
pca = PCA(n_components=2)
X_2d = pca.fit_transform(X)
print(f"Reduced shape: {X_2d.shape}") # (1797, 2)
print(f"Variance explained: {pca.explained_variance_ratio_.sum():.2%}")
# Find optimal number of components
pca_full = PCA().fit(X)
cumulative_variance = np.cumsum(pca_full.explained_variance_ratio_)
n_components_95 = np.argmax(cumulative_variance >= 0.95) + 1
print(f"Components needed for 95% variance: {n_components_95}")Anomaly Detection
Example
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
import numpy as np
# Normal data + some anomalies
np.random.seed(42)
X_normal = np.random.randn(200, 2)
X_anomaly = np.random.uniform(low=-4, high=4, size=(10, 2))
X = np.vstack([X_normal, X_anomaly])
# Isolation Forest
iso = IsolationForest(contamination=0.05, random_state=42)
iso_labels = iso.fit_predict(X)
print(f"Isolation Forest detected {sum(iso_labels == -1)} anomalies")
# Local Outlier Factor
lof = LocalOutlierFactor(contamination=0.05)
lof_labels = lof.fit_predict(X)
print(f"LOF detected {sum(lof_labels == -1)} anomalies")
Real-World Applications
| Application | Technique | Business Value |
|---|---|---|
| Customer segmentation | K-Means, GMM | Targeted marketing campaigns |
| Fraud detection | Isolation Forest | Catch unusual transactions |
| Document organization | Topic modeling | Auto-categorize content |
| Image compression | PCA, Autoencoders | Reduce storage costs |
| Social network analysis | Community detection | Identify user groups |
| Market basket analysis | Apriori | Cross-selling recommendations |
Customer Segmentation Example
Example
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd
# Simulate customer data
np.random.seed(42)
customers = pd.DataFrame({
'annual_income': np.concatenate([
np.random.normal(30000, 5000, 50),
np.random.normal(60000, 8000, 50),
np.random.normal(100000, 15000, 50)
]),
'spending_score': np.concatenate([
np.random.normal(30, 10, 50),
np.random.normal(60, 15, 50),
np.random.normal(80, 10, 50)
]),
'age': np.concatenate([
np.random.normal(45, 10, 50),
np.random.normal(35, 8, 50),
np.random.normal(30, 5, 50)
])
})
# Scale features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(customers)
# Find optimal k using elbow method
inertias = []
for k in range(1, 11):
km = KMeans(n_clusters=k, random_state=42, n_init=10)
km.fit(X_scaled)
inertias.append(km.inertia_)
# Apply K-Means with optimal k
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
customers['segment'] = kmeans.fit_predict(X_scaled)
# Analyze segments
print("Customer Segments:")
print(customers.groupby('segment').mean().round(0))
Evaluating Unsupervised Models
Unlike supervised learning, there's no "correct answer" to compare against:
Example
from sklearn.metrics import silhouette_score, calinski_harabasz_score, davies_bouldin_score
# Internal evaluation metrics (no ground truth needed)
labels = kmeans.predict(X_scaled)
silhouette = silhouette_score(X_scaled, labels)
calinski = calinski_harabasz_score(X_scaled, labels)
davies = davies_bouldin_score(X_scaled, labels)
print("Clustering Quality Metrics:")
print(f" Silhouette Score: {silhouette:.4f} (higher = better, range [-1, 1])")
print(f" Calinski-Harabasz Score: {calinski:.1f} (higher = better)")
print(f" Davies-Bouldin Score: {davies:.4f} (lower = better)")
Interview Questions
- How do you evaluate unsupervised learning models without labels?
Use internal metrics like silhouette score, elbow method for clustering, or reconstruction error for dimensionality reduction. Also validate with domain experts and business logic.
- When would you choose DBSCAN over K-Means?
When clusters have irregular shapes, when you don't know the number of clusters in advance, or when you need to identify noise points (outliers).
- What is the difference between PCA and t-SNE?
PCA is linear, preserves global structure, fast, and invertible. t-SNE is non-linear, preserves local structure, slower, and mainly used for visualization (not feature reduction).
- How do you determine the right number of clusters?
Use the elbow method (inertia plot), silhouette analysis, gap statistic, or domain knowledge. There's often no single "right" answer.
- Can unsupervised learning be used as a preprocessing step for supervised learning?
Yes! Clustering can create new features, PCA can reduce dimensionality before classification, and anomaly detection can clean data before training.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Unsupervised Learning.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Machine Learning topic.
Search Terms
machine-learning, machine learning, machine, learning, fundamentals, unsupervised, unsupervised learning
Related Machine Learning Topics
Continue learning this concept
Machine Learning FundamentalsBias vs Variance TradeoffMaster the bias-variance tradeoff in machine learning with intuitive explanations, mathematical formulations, visual examples, and practical strategies to balance model complexity.Machine Learning FundamentalsModel Evaluation OverviewComplete overview of model evaluation strategies in machine learning including metrics for classification and regression, validation techniques, and comparison methods.Machine Learning FundamentalsOverfitting and UnderfittingLearn to identify and prevent overfitting and underfitting in machine learning models with practical techniques, code examples, and regularization strategies.Machine Learning FundamentalsReinforcement Learning Fundamentals — Machine LearningIntroduction to reinforcement learning concepts including agents, environments, rewards, policies, and value functions with Python examples.Machine Learning FundamentalsSupervised LearningDeep dive into supervised learning covering classification, regression, common algorithms, training process, and practical Python implementations with sklearn.Machine Learning FundamentalsTraining vs Testing DataUnderstanding the importance of splitting data into training, validation, and test sets with proper techniques, ratios, and common mistakes to avoid.