ML Notes
HubAI vs ML vs Deep Learning — Machine LearningApplications of Machine LearningHistory of Machine LearningMachine Learning RoadmapWhat is Machine Learning?Calculus for Machine LearningData Preprocessing BasicsLinear Algebra for Machine LearningProbability Basics for Machine LearningPython for Machine LearningStatistics Basics for Machine LearningBias vs Variance TradeoffModel Evaluation OverviewOverfitting and UnderfittingReinforcement Learning Fundamentals — Machine LearningSupervised LearningTraining vs Testing DataTypes of Machine LearningUnsupervised LearningData Cleaning — Machine LearningData Collection for Machine LearningData Visualization for MLEncoding Categorical DataFeature EngineeringFeature ScalingHandling Missing ValuesTrain-Test SplitElastic Net RegressionLasso Regression (L1 Regularization)Multiple Linear RegressionPolynomial RegressionIntroduction to RegressionRegression ProjectsRidge Regression (L2 Regularization)Simple Linear RegressionIntroduction to ClassificationClassification ProjectsDecision TreesK-Nearest Neighbors (KNN)Logistic RegressionNaive Bayes ClassifierRandom Forest ClassificationSupport Vector Machine (SVM)XGBoost for ClassificationClustering in Machine Learning: Complete Guide to Unsupervised LearningClustering Projects: Real-World Applications and Implementation GuidesDBSCAN ClusteringGaussian Mixture Models (GMM): Probabilistic Clustering for Advanced Data AnalysisHierarchical ClusteringK-Means ClusteringDimensionality Reduction: Comprehensive Guide to Feature Reduction TechniquesFeature Selection Techniques: Choosing Informative Variables for ML ModelsLinear Discriminant Analysis (LDA): Supervised Dimensionality ReductionPrincipal Component Analysis (PCA)t-SNE: Non-Linear Dimensionality Reduction for Data VisualizationUMAP: Uniform Manifold Approximation and Projection for Modern Data VisualizationAdaBoost: Adaptive Boosting Algorithm Explained with ExamplesBagging: Bootstrap Aggregating for Reducing Model VarianceBoosting: Sequential Ensemble Learning for Bias and Variance ReductionEnsemble Learning: Complete Guide to Combining ML Models for Better PredictionsGradient Boosting: Building Powerful ML Models Through Sequential OptimizationRandom Forest: Complete Guide to Tree Ensembles and Feature ImportanceStacking: Advanced Ensemble Method Using Meta-Learners for Superior ML PredictionsActivation Functions in Neural NetworksBackpropagation: Complete Algorithm for Training Neural NetworksIntroduction to Deep Learning: Neural Networks, Architectures, and ApplicationsDeep Learning Projects: Hands-On Applications and Case StudiesNeural Networks Basics: Perceptron, Architecture, and TrainingPerceptron: Foundational Single-Neuron Classifier for Binary ClassificationPyTorch Basics: Building and Training Neural NetworksTensorFlow Basics: Building and Training Neural NetworksIntroduction to Natural Language Processing: Fundamentals and ApplicationsNLP Projects: Practical Applications from Sentiment Analysis to Machine TranslationStemming and Lemmatization: Reducing Words to Root Forms in NLPText Preprocessing: Essential Steps for NLP ApplicationsTF-IDF: Term Frequency-Inverse Document Frequency for Text Feature ExtractionTokenization in NLP: Breaking Text into Meaningful UnitsTransformers: The Architecture Behind Modern NLP ModelsWord Embeddings: Representing Words as Vectors for NLPIntroduction to Computer VisionComputer Vision ProjectsImage Classification: Building and Deploying CNN ModelsImage Processing BasicsImage Segmentation — Machine LearningObject DetectionOpenCV BasicsAgents and Environments in RLDeep Q-Networks (DQN)Policy Gradient Methods — Machine LearningQ-Learning AlgorithmReinforcement Learning: IntroductionReinforcement Learning ProjectsAccuracy, Precision, Recall: Classification Metrics Explained for ML InterviewConfusion Matrix: Complete Guide to Classification EvaluationCross-Validation: Comprehensive Guide to Robust Model EvaluationF1 Score: Balanced Classification Metric for Imbalanced DataGrid Search: Complete Guide to Hyperparameter Tuning in Machine LearningHyperparameter Tuning: Complete Guide to Optimizing Machine Learning ModelsModel Selection: Choosing the Best Algorithm for Your ML ProblemROC and AUC: Complete Guide to Classification Threshold EvaluationBuild an AI Assistant with LLMsBuild a Conversational ChatbotCustomer Churn PredictionFace Recognition System — Machine LearningHouse Price PredictionRecommendation System ProjectSentiment Analysis Project — Machine LearningSpam Email ClassifierML Case Study Questions for InterviewsML Coding Questions for Technical InterviewsMachine Learning Interview CheatsheetEssential Machine Learning Interview QuestionsStatistics for Machine Learning InterviewsDatasets for ML PracticeImportant ML FormulasMachine Learning GlossaryRecommended ML Books and CoursesCI/CD for Machine LearningDocker for Machine LearningFastAPI for Machine Learning APIsFlask API for Machine LearningMLOps: IntroductionModel Deployment StrategiesML Model Monitoring & Observability
Complete strategies for handling missing data in ML including imputation techniques, deletion methods, and advanced approaches like KNN and iterative imputation.
Missing data is one of the most common challenges in real-world ML projects. How you handle it can significantly impact model performance. The wrong approach can introduce bias, while the right approach can actually improve your results.
Types of Missing Data
| │ | No pattern; data is randomly missing │ |
| │ | Example: Sensor randomly fails │ |
| │ | Missingness depends on OBSERVED variables │ |
| │ | Example: Young people skip income question │ |
| │ | Missingness depends on the MISSING value itself │ |
| │ | Example: High-income people hide their salary │ |
Detecting Missing Values
Example
import pandas as pd
import numpy as np
# Create dataset with various missing patterns
np.random.seed(42)
n = 1000
df = pd.DataFrame({
'age': np.random.normal(35, 10, n),
'income': np.random.normal(50000, 15000, n),
'education_years': np.random.randint(8, 22, n).astype(float),
'credit_score': np.random.normal(700, 50, n),
})
# Introduce missing values with different patterns
df.loc[np.random.choice(n, 50, replace=False), 'age'] = np.nan # MCAR
df.loc[df['age'] < 25, 'income'] = np.nan # MAR (young people missing income)
df.loc[df['credit_score'] > 750, 'credit_score'] = np.nan # MNAR
# Missing value analysis
print("Missing Value Summary:")
print("-" * 50)
missing = df.isnull().sum()
missing_pct = (df.isnull().sum() / len(df) * 100).round(2)
summary = pd.DataFrame({'Count': missing, 'Percentage': missing_pct})
print(summary[summary['Count'] > 0])
print(f"\nTotal rows with any missing: {df.isnull().any(axis=1).sum()}")
Strategy 1: Deletion
Example
# 1a. Listwise deletion (drop rows)
df_dropped_rows = df.dropna()
print(f"After dropping rows: {len(df_dropped_rows)}/{len(df)} rows remain")
# 1b. Drop columns with too many missing values
threshold = 0.5 # Drop if >50% missing
cols_to_keep = df.columns[df.isnull().mean() < threshold]
df_dropped_cols = df[cols_to_keep]
print(f"Columns kept: {list(cols_to_keep)}")
# When to use deletion:
# - Very few missing values (<5%)
# - Large dataset where losing rows doesn't matter
# - Missing Completely At Random (MCAR)
Strategy 2: Simple Imputation
Example
from sklearn.impute import SimpleImputer
import pandas as pd
import numpy as np
# Mean imputation (for normally distributed data)
mean_imputer = SimpleImputer(strategy='mean')
df_mean = pd.DataFrame(
mean_imputer.fit_transform(df), columns=df.columns
)
# Median imputation (robust to outliers)
median_imputer = SimpleImputer(strategy='median')
df_median = pd.DataFrame(
median_imputer.fit_transform(df), columns=df.columns
)
# Mode imputation (for categorical data)
# For categorical: SimpleImputer(strategy='most_frequent')
# Constant imputation
constant_imputer = SimpleImputer(strategy='constant', fill_value=-999)
print("Imputation Comparison (Income column):")
print(f" Original mean: {df['income'].mean():.0f}")
print(f" After mean imp: {df_mean['income'].mean():.0f}")
print(f" Original std: {df['income'].std():.0f}")
print(f" After mean imp: {df_mean['income'].std():.0f}")
print(" ⚠️ Mean imputation reduces variance!")
Strategy 3: Advanced Imputation
Example
from sklearn.impute import KNNImputer
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer
# KNN Imputer - uses similar samples to fill missing values
knn_imputer = KNNImputer(n_neighbors=5)
df_knn = pd.DataFrame(
knn_imputer.fit_transform(df), columns=df.columns
)
# Iterative Imputer (MICE - Multiple Imputation by Chained Equations)
iter_imputer = IterativeImputer(max_iter=10, random_state=42)
df_iter = pd.DataFrame(
iter_imputer.fit_transform(df), columns=df.columns
)
print("Advanced Imputation Results:")
print(f" KNN preserved std better: {df_knn['income'].std():.0f}")
print(f" Iterative preserved std: {df_iter['income'].std():.0f}")
print(f" Original std: {df['income'].std():.0f}")
Strategy 4: Indicator Variables
Example
# Create a binary column indicating whether data was missing
df_with_indicator = df.copy()
for col in df.columns:
if df[col].isnull().any():
df_with_indicator[f'{col}_was_missing'] = df[col].isnull().astype(int)
# Then impute the original column
df_with_indicator = df_with_indicator.fillna(df_with_indicator.median())
print("Columns after adding missing indicators:")
print(df_with_indicator.columns.tolist())
print("\nThis preserves information about WHY data was missing!")
Choosing the Right Strategy
| Situation | Recommended Approach |
|---|---|
| < 5% missing, MCAR | Delete rows |
| Numerical, few missing | Median imputation |
| Categorical | Mode or constant ("Unknown") |
| Complex patterns | KNN or Iterative imputer |
| Missingness is informative | Add indicator variable |
| Time series | Forward/backward fill |
Interview Questions
- What are the types of missing data mechanisms?
MCAR (random), MAR (depends on observed data), MNAR (depends on missing value itself). The mechanism determines which imputation strategy is appropriate.
- Why is mean imputation problematic?
It reduces variance, distorts correlations between features, and doesn't account for uncertainty. It's like saying everyone is "average" — unrealistic.
- When would you use KNN imputation over simple imputation?
When features are correlated and missing values can be estimated from similar data points. KNN preserves relationships between features better than mean/median.
- How do you handle missing values in production (during inference)?
Use the same imputation strategy and parameters learned from training data. Store the imputer object and apply it consistently to new data.
- Should you impute before or after train-test split?
AFTER splitting. Imputation parameters (mean, median, KNN neighbors) must be learned only from training data to prevent data leakage.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Handling Missing Values.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Machine Learning topic.
Search Terms
machine-learning, machine learning, machine, learning, data, preprocessing, handling, missing
Related Machine Learning Topics
Continue learning this concept
Data PreprocessingData Cleaning — Machine LearningComplete guide to data cleaning for ML including handling duplicates, fixing data types, dealing with inconsistencies, and automated cleaning pipelines.Data PreprocessingData Collection for Machine LearningLearn effective data collection strategies for ML projects including web scraping, APIs, databases, surveys, and data quality assessment techniques.Data PreprocessingData Visualization for MLEssential data visualization techniques for machine learning including EDA plots, feature analysis, distribution analysis, and correlation visualization with Python.Data PreprocessingEncoding Categorical DataComplete guide to encoding categorical variables for ML including one-hot encoding, label encoding, target encoding, and handling high-cardinality features.Data PreprocessingFeature EngineeringMaster feature engineering for ML including creating new features, polynomial features, interaction terms, domain-specific features, and automated feature generation.Data PreprocessingFeature ScalingComplete guide to feature scaling techniques including standardization, normalization, and when to use each method with practical sklearn implementations.