ML Notes
HubAI vs ML vs Deep Learning — Machine LearningApplications of Machine LearningHistory of Machine LearningMachine Learning RoadmapWhat is Machine Learning?Calculus for Machine LearningData Preprocessing BasicsLinear Algebra for Machine LearningProbability Basics for Machine LearningPython for Machine LearningStatistics Basics for Machine LearningBias vs Variance TradeoffModel Evaluation OverviewOverfitting and UnderfittingReinforcement Learning Fundamentals — Machine LearningSupervised LearningTraining vs Testing DataTypes of Machine LearningUnsupervised LearningData Cleaning — Machine LearningData Collection for Machine LearningData Visualization for MLEncoding Categorical DataFeature EngineeringFeature ScalingHandling Missing ValuesTrain-Test SplitElastic Net RegressionLasso Regression (L1 Regularization)Multiple Linear RegressionPolynomial RegressionIntroduction to RegressionRegression ProjectsRidge Regression (L2 Regularization)Simple Linear RegressionIntroduction to ClassificationClassification ProjectsDecision TreesK-Nearest Neighbors (KNN)Logistic RegressionNaive Bayes ClassifierRandom Forest ClassificationSupport Vector Machine (SVM)XGBoost for ClassificationClustering in Machine Learning: Complete Guide to Unsupervised LearningClustering Projects: Real-World Applications and Implementation GuidesDBSCAN ClusteringGaussian Mixture Models (GMM): Probabilistic Clustering for Advanced Data AnalysisHierarchical ClusteringK-Means ClusteringDimensionality Reduction: Comprehensive Guide to Feature Reduction TechniquesFeature Selection Techniques: Choosing Informative Variables for ML ModelsLinear Discriminant Analysis (LDA): Supervised Dimensionality ReductionPrincipal Component Analysis (PCA)t-SNE: Non-Linear Dimensionality Reduction for Data VisualizationUMAP: Uniform Manifold Approximation and Projection for Modern Data VisualizationAdaBoost: Adaptive Boosting Algorithm Explained with ExamplesBagging: Bootstrap Aggregating for Reducing Model VarianceBoosting: Sequential Ensemble Learning for Bias and Variance ReductionEnsemble Learning: Complete Guide to Combining ML Models for Better PredictionsGradient Boosting: Building Powerful ML Models Through Sequential OptimizationRandom Forest: Complete Guide to Tree Ensembles and Feature ImportanceStacking: Advanced Ensemble Method Using Meta-Learners for Superior ML PredictionsActivation Functions in Neural NetworksBackpropagation: Complete Algorithm for Training Neural NetworksIntroduction to Deep Learning: Neural Networks, Architectures, and ApplicationsDeep Learning Projects: Hands-On Applications and Case StudiesNeural Networks Basics: Perceptron, Architecture, and TrainingPerceptron: Foundational Single-Neuron Classifier for Binary ClassificationPyTorch Basics: Building and Training Neural NetworksTensorFlow Basics: Building and Training Neural NetworksIntroduction to Natural Language Processing: Fundamentals and ApplicationsNLP Projects: Practical Applications from Sentiment Analysis to Machine TranslationStemming and Lemmatization: Reducing Words to Root Forms in NLPText Preprocessing: Essential Steps for NLP ApplicationsTF-IDF: Term Frequency-Inverse Document Frequency for Text Feature ExtractionTokenization in NLP: Breaking Text into Meaningful UnitsTransformers: The Architecture Behind Modern NLP ModelsWord Embeddings: Representing Words as Vectors for NLPIntroduction to Computer VisionComputer Vision ProjectsImage Classification: Building and Deploying CNN ModelsImage Processing BasicsImage Segmentation — Machine LearningObject DetectionOpenCV BasicsAgents and Environments in RLDeep Q-Networks (DQN)Policy Gradient Methods — Machine LearningQ-Learning AlgorithmReinforcement Learning: IntroductionReinforcement Learning ProjectsAccuracy, Precision, Recall: Classification Metrics Explained for ML InterviewConfusion Matrix: Complete Guide to Classification EvaluationCross-Validation: Comprehensive Guide to Robust Model EvaluationF1 Score: Balanced Classification Metric for Imbalanced DataGrid Search: Complete Guide to Hyperparameter Tuning in Machine LearningHyperparameter Tuning: Complete Guide to Optimizing Machine Learning ModelsModel Selection: Choosing the Best Algorithm for Your ML ProblemROC and AUC: Complete Guide to Classification Threshold EvaluationBuild an AI Assistant with LLMsBuild a Conversational ChatbotCustomer Churn PredictionFace Recognition System — Machine LearningHouse Price PredictionRecommendation System ProjectSentiment Analysis Project — Machine LearningSpam Email ClassifierML Case Study Questions for InterviewsML Coding Questions for Technical InterviewsMachine Learning Interview CheatsheetEssential Machine Learning Interview QuestionsStatistics for Machine Learning InterviewsDatasets for ML PracticeImportant ML FormulasMachine Learning GlossaryRecommended ML Books and CoursesCI/CD for Machine LearningDocker for Machine LearningFastAPI for Machine Learning APIsFlask API for Machine LearningMLOps: IntroductionModel Deployment StrategiesML Model Monitoring & Observability
Quick reference guide for ML interview prep with algorithms, formulas, time complexities, and key concepts.
A comprehensive quick reference for machine learning interviews. Print this out, memorize key concepts, and reference it when practicing.
Algorithm Complexity Cheatsheet
| Algorithm | Time Complexity | Space | Best For |
|---|---|---|---|
| Linear Regression | O(n*d) | O(d) | Small datasets, interpretability |
| Logistic Regression | O(n*d*i) | O(d) | Binary classification |
| K-Means | O(n*k*i*d) | O(n+k*d) | Clustering, unlabeled data |
| KNN | O(n*d) predict | O(n*d) | Non-linear boundaries |
| Decision Tree | O(n*d*log n) | O(log n) depth | Interpretability, non-linear |
| Random Forest | O(n*m*log n) | O(m*log n) | High accuracy, generalization |
| SVM | O(n²) to O(n³) | O(n) | Small-medium datasets |
| Gradient Boosting | O(n*m*i) | O(m) | Best performance, competitions |
| Neural Networks | O(n*h*i) | O(h) | Complex patterns, large data |
| K-NN Search | O(n*d) naive, O(log n) with tree | O(n*d) | Fast retrieval |
Key Metrics Reference
Classification Metrics
Example
TP = True Positive (predicted 1, actual 1)
FP = False Positive (predicted 1, actual 0)
TN = True Negative (predicted 0, actual 0)
FN = False Negative (predicted 0, actual 1)
Accuracy = (TP + TN) / (TP + FP + TN + FN)
Precision = TP / (TP + FP) [accuracy of positive predictions]
Recall = TP / (TP + FN) [coverage of actual positives]
F1 = 2 * (Precision * Recall) / (Precision + Recall)
Specificity = TN / (TN + FP) [recall for negative class]
ROC-AUC: Area under curve of TPR vs FPR
TPR = TP / (TP + FN) [True Positive Rate]
FPR = FP / (FP + TN) [False Positive Rate]
Regression Metrics
Example
MSE = Mean Squared Error = (1/n) * Σ(y_true - y_pred)²
RMSE = √MSE
MAE = Mean Absolute Error = (1/n) * Σ|y_true - y_pred|
R² = 1 - (SS_res / SS_tot)
SS_res = Σ(y_true - y_pred)²
SS_tot = Σ(y_true - y_mean)²
Range: 0 to 1 (higher is better)
Clustering Metrics
| Silhouette Score | measures how similar object is to own cluster vs others |
| Range | -1 to 1 (higher is better) |
| Davies-Bouldin Index | ratio of within to between cluster distances |
| Range | 0 to ∞ (lower is better) |
| Inertia | sum of squared distances to nearest centroid (for K-Means) |
Model Selection Guide
When to Use Each Algorithm
| Problem | Algorithm | Reason |
|---|---|---|
| High-dimensional data | Linear models, SVM | Curse of dimensionality |
| Non-linear boundaries | Random Forest, Neural Nets | Can learn complex patterns |
| Imbalanced classification | XGBoost with scale_pos_weight | Handles class imbalance well |
| Large dataset (>1M rows) | Gradient Boosting, Neural Nets | Scalable training |
| Need interpretability | Linear models, Decision Trees | Easy to explain |
| Fast inference needed | Decision Trees, SVM | Low latency serving |
| Few features, many samples | Linear Regression | Simple is better |
| Many features, few samples | Regularized models (Ridge/Lasso) | Prevents overfitting |
Bias-Variance Tradeoff
| - Solution | more features, more complex model |
| - Symptoms | High training error |
| - Solution | regularization, more data, cross-validation |
| - Symptoms | Low training error, high test error |
Regularization Techniques
L1 Regularization (Lasso)
loss = MSE + λ * Σ|weights|
Effect: Feature selection (some weights → 0)
L2 Regularization (Ridge)
loss = MSE + λ * Σ(weights²)
Effect: Weight shrinkage, no feature elimination
Elastic Net
loss = MSE + λ1*Σ|w| + λ2*Σ(w²)
Effect: Combination of L1 and L2
Early Stopping
- Monitor validation loss during training
- Stop when validation loss increases
- Prevents overfitting without explicit penalty
Feature Engineering Checklist
| - StandardScaler | (x - mean) / std |
| - MinMaxScaler | (x - min) / (max - min) |
| - RobustScaler | (x - median) / IQR |
| - Polynomial features | x1*x2, x1², etc |
| - Ratio features | x1/x2 |
| - IQR method | remove if x < Q1 - 1.5*IQR |
| - Z-score | remove if |z| > 3 |
Cross-Validation Strategies
K-Fold Cross-Validation (most common)
- Split data into k folds
- Train on k-1 folds, test on 1
- Repeat k times
- Report mean ± std of scores
Stratified K-Fold
- Maintains class distribution in each fold
- Better for imbalanced datasets
Time Series Cross-Validation
- Walk-forward: train on past, test on future
- Respects temporal ordering
Leave-One-Out (LOO)
- Each sample is test set once
- Only for small datasets (computationally expensive)
Hyperparameter Tuning
Grid Search
- Test all combinations
- Exhaustive but slow
- Best for small parameter spaces
Random Search
- Sample random combinations
- Often as good as grid search
- Faster for large spaces
Bayesian Optimization
- Uses Gaussian processes
- Smart sampling based on previous results
- Most efficient for expensive functions
Common Hyperparameters by Algorithm
├── Linear Models: regularization strength (C, λ)
├── Trees: max_depth, min_samples_split, min_samples_leaf
├── Ensemble: n_estimators, learning_rate, max_depth
├── SVM: C (regularization), kernel type, kernel parameters
├── Neural Networks: layers, units, learning_rate, batch_size, epochs
└── Clustering: k (number of clusters), distance metric
Handling Class Imbalance
| ├── Oversampling | duplicate minority class |
| ├── Undersampling | reduce majority class |
| ├── SMOTE | generate synthetic minority samples |
| ├── Default threshold | 0.5 |
Data Preparation Workflow
1. Load & Explore
├── Data shape and types
├── Missing values
├── Descriptive statistics
└── Visualizations
2. Handle Missing Values
├── Identify patterns
├── Choose imputation strategy
└── Validate imputation
3. Outlier Treatment
├── Identify outliers
├── Decide: remove, cap, or keep
└── Document decisions
4. Feature Engineering
├── Create new features
├── Encode categorical
├── Scale numerical
└── Feature selection
5. Data Split
├── Stratified split for classification
├── Temporal split for time series
└── Typically 70-80% train, 20-30% test
6. Baseline Model
├── Simple model first
├── Measure performance
└── Use as benchmark
Neural Network Basics
Forward Pass
z = w*x + b (linear transformation)
a = activation(z) (non-linearity)
output = w_out*a + b_out (output layer)
Backpropagation
∂L/∂w = ∂L/∂a * ∂a/∂z * ∂z/∂w (chain rule)
Weight Update (Gradient Descent)
w_new = w_old - learning_rate * ∂L/∂w
Common Activation Functions
├── ReLU: max(0, x) [hidden layers, default]
├── Sigmoid: 1/(1+e^-x) [binary classification]
├── Tanh: (e^x - e^-x)/(e^x + e^-x) [-1 to 1 range]
└── Softmax: e^x_i / Σ(e^x) [multi-class, output]
Optimizers
├── SGD: w = w - lr*gradient
├── Momentum: accumulate gradient history
├── Adam: adaptive learning rates per parameter
└── RMSprop: root mean square propagation
Model Evaluation Checklist
Code example
□ Training Curve Analysis
- Plot training vs validation loss
- Look for overfitting/underfitting
- Check if loss is decreasing
□ Cross-Validation
- Use stratified k-fold
- Report mean ± std of metrics
- Check if folds have similar performance
□ Class Balance Check
- For classification: check class distribution
- In train, validation, test sets
- Use stratified split if imbalanced
□ Feature Importance
- Identify which features matter
- Remove uninformative features
- Check for data leakage
□ Error Analysis
- Which samples does model get wrong?
- Any patterns in errors?
- Could indicate data quality issues
□ Robustness Testing
- Test on different data distributions
- Stress test with edge cases
- Check sensitivity to input perturbationsCommon Interview Mistakes to Avoid
| Mistake | Fix |
|---|---|
| Not splitting train/test before preprocessing | Apply preprocessing to each fold separately |
| Using accuracy for imbalanced data | Use precision, recall, F1, AUC-ROC |
| Tuning hyperparameters on test set | Use validation set for tuning |
| Not handling categorical variables | One-hot encode or label encode before modeling |
| Forgetting to scale features | Always scale for distance-based algorithms |
| No baseline model | Compare new model to simple baseline |
| Overfitting to small dataset | Use cross-validation, regularization |
| Ignoring data quality | Understand and clean data first |
| Not testing edge cases | Test with missing values, outliers, etc |
Quick Python Snippets
python example
# Standard ML workflow
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
# 1. Load and split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# 2. Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# 3. Train model
model = RandomForestClassifier(n_estimators=100, max_depth=10)
model.fit(X_train, y_train)
# 4. Evaluate
train_score = model.score(X_train, y_train)
test_score = model.score(X_test, y_test)
cv_scores = cross_val_score(model, X_train, y_train, cv=5)
print(f"Train: {train_score:.4f}")
print(f"Test: {test_score:.4f}")
print(f"CV: {cv_scores.mean():.4f} ± {cv_scores.std():.4f}")
print(classification_report(y_test, model.predict(X_test)))Summary
Memorize:
- Key algorithm complexities
- Metric formulas and when to use them
- Bias-variance tradeoff
- Common hyperparameters
- Data preparation workflow
Practice implementing algorithms from scratch to really understand them. Good luck with your interviews!
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Machine Learning Interview Cheatsheet.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Machine Learning topic.
Search Terms
machine-learning, machine learning, machine, learning, interview, preparation, cheatsheet, machine learning interview cheatsheet
Related Machine Learning Topics
Continue learning this concept
Interview PreparationML Case Study Questions for InterviewsComprehensive collection of real-world machine learning case study questions asked in interviews at top tech companies like Google, Meta, Amazon, and Microsoft.Interview PreparationML Coding Questions for Technical InterviewsEssential coding problems for machine learning interviews including data manipulation, algorithm implementation, and optimization challenges from top tech companies.Interview PreparationEssential Machine Learning Interview QuestionsComprehensive list of technical ML interview questions from companies like Google, Meta, Amazon with detailed answers and explanations.Interview PreparationStatistics for Machine Learning InterviewsEssential statistics concepts for ML interviews: hypothesis testing, distributions, sampling, Bayesian thinking, and probability theory.ClassificationIntroduction to ClassificationComplete introduction to classification in machine learning covering binary and multi-class problems, decision boundaries, and algorithm selection strategies.ClassificationClassification ProjectsHands-on classification projects covering spam detection, disease prediction, image classification, and customer churn with complete Python implementations.