ML Notes
HubAI vs ML vs Deep Learning — Machine LearningApplications of Machine LearningHistory of Machine LearningMachine Learning RoadmapWhat is Machine Learning?Calculus for Machine LearningData Preprocessing BasicsLinear Algebra for Machine LearningProbability Basics for Machine LearningPython for Machine LearningStatistics Basics for Machine LearningBias vs Variance TradeoffModel Evaluation OverviewOverfitting and UnderfittingReinforcement Learning Fundamentals — Machine LearningSupervised LearningTraining vs Testing DataTypes of Machine LearningUnsupervised LearningData Cleaning — Machine LearningData Collection for Machine LearningData Visualization for MLEncoding Categorical DataFeature EngineeringFeature ScalingHandling Missing ValuesTrain-Test SplitElastic Net RegressionLasso Regression (L1 Regularization)Multiple Linear RegressionPolynomial RegressionIntroduction to RegressionRegression ProjectsRidge Regression (L2 Regularization)Simple Linear RegressionIntroduction to ClassificationClassification ProjectsDecision TreesK-Nearest Neighbors (KNN)Logistic RegressionNaive Bayes ClassifierRandom Forest ClassificationSupport Vector Machine (SVM)XGBoost for ClassificationClustering in Machine Learning: Complete Guide to Unsupervised LearningClustering Projects: Real-World Applications and Implementation GuidesDBSCAN ClusteringGaussian Mixture Models (GMM): Probabilistic Clustering for Advanced Data AnalysisHierarchical ClusteringK-Means ClusteringDimensionality Reduction: Comprehensive Guide to Feature Reduction TechniquesFeature Selection Techniques: Choosing Informative Variables for ML ModelsLinear Discriminant Analysis (LDA): Supervised Dimensionality ReductionPrincipal Component Analysis (PCA)t-SNE: Non-Linear Dimensionality Reduction for Data VisualizationUMAP: Uniform Manifold Approximation and Projection for Modern Data VisualizationAdaBoost: Adaptive Boosting Algorithm Explained with ExamplesBagging: Bootstrap Aggregating for Reducing Model VarianceBoosting: Sequential Ensemble Learning for Bias and Variance ReductionEnsemble Learning: Complete Guide to Combining ML Models for Better PredictionsGradient Boosting: Building Powerful ML Models Through Sequential OptimizationRandom Forest: Complete Guide to Tree Ensembles and Feature ImportanceStacking: Advanced Ensemble Method Using Meta-Learners for Superior ML PredictionsActivation Functions in Neural NetworksBackpropagation: Complete Algorithm for Training Neural NetworksIntroduction to Deep Learning: Neural Networks, Architectures, and ApplicationsDeep Learning Projects: Hands-On Applications and Case StudiesNeural Networks Basics: Perceptron, Architecture, and TrainingPerceptron: Foundational Single-Neuron Classifier for Binary ClassificationPyTorch Basics: Building and Training Neural NetworksTensorFlow Basics: Building and Training Neural NetworksIntroduction to Natural Language Processing: Fundamentals and ApplicationsNLP Projects: Practical Applications from Sentiment Analysis to Machine TranslationStemming and Lemmatization: Reducing Words to Root Forms in NLPText Preprocessing: Essential Steps for NLP ApplicationsTF-IDF: Term Frequency-Inverse Document Frequency for Text Feature ExtractionTokenization in NLP: Breaking Text into Meaningful UnitsTransformers: The Architecture Behind Modern NLP ModelsWord Embeddings: Representing Words as Vectors for NLPIntroduction to Computer VisionComputer Vision ProjectsImage Classification: Building and Deploying CNN ModelsImage Processing BasicsImage Segmentation — Machine LearningObject DetectionOpenCV BasicsAgents and Environments in RLDeep Q-Networks (DQN)Policy Gradient Methods — Machine LearningQ-Learning AlgorithmReinforcement Learning: IntroductionReinforcement Learning ProjectsAccuracy, Precision, Recall: Classification Metrics Explained for ML InterviewConfusion Matrix: Complete Guide to Classification EvaluationCross-Validation: Comprehensive Guide to Robust Model EvaluationF1 Score: Balanced Classification Metric for Imbalanced DataGrid Search: Complete Guide to Hyperparameter Tuning in Machine LearningHyperparameter Tuning: Complete Guide to Optimizing Machine Learning ModelsModel Selection: Choosing the Best Algorithm for Your ML ProblemROC and AUC: Complete Guide to Classification Threshold EvaluationBuild an AI Assistant with LLMsBuild a Conversational ChatbotCustomer Churn PredictionFace Recognition System — Machine LearningHouse Price PredictionRecommendation System ProjectSentiment Analysis Project — Machine LearningSpam Email ClassifierML Case Study Questions for InterviewsML Coding Questions for Technical InterviewsMachine Learning Interview CheatsheetEssential Machine Learning Interview QuestionsStatistics for Machine Learning InterviewsDatasets for ML PracticeImportant ML FormulasMachine Learning GlossaryRecommended ML Books and CoursesCI/CD for Machine LearningDocker for Machine LearningFastAPI for Machine Learning APIsFlask API for Machine LearningMLOps: IntroductionModel Deployment StrategiesML Model Monitoring & Observability
Collection of essential mathematical formulas used in machine learning algorithms and evaluation.
Quick reference for key mathematical formulas in machine learning.
Linear Regression
Hypothesis Function:
Example
h(x) = θ₀ + θ₁x₁ + θ₂x₂ + ... + θₙxₙ = θᵀx
Cost Function (MSE):
Example
J(θ) = (1/2m) Σᵢ₌₁ᵐ (hθ(xⁱ) - yⁱ)²
m = number of training samples
hθ(x) = predicted value
y = actual value
Gradient Descent Update:
Example
θⱼ := θⱼ - α * (∂J/∂θⱼ)
α = learning rate
∂J/∂θⱼ = (1/m) Σᵢ₌₁ᵐ (hθ(xⁱ) - yⁱ) * xⱼⁱ
Logistic Regression
Sigmoid Function:
σ(z) = 1 / (1 + e^(-z))
Maps output to [0, 1] for probability
Hypothesis:
hθ(x) = σ(θᵀx) = 1 / (1 + e^(-θᵀx))
Cost Function (Log Loss):
Example
J(θ) = -(1/m) Σᵢ₌₁ᵐ [yⁱ log(hθ(xⁱ)) + (1 - yⁱ) log(1 - hθ(xⁱ))]
y ∈ {0, 1} (binary classification)
Gradient:
Example
∂J/∂θⱼ = (1/m) Σᵢ₌₁ᵐ (hθ(xⁱ) - yⁱ) * xⱼⁱ
Regularization
L1 Regularization (Lasso):
Example
J(θ) = J₀(θ) + (λ/2m) Σⱼ₌₁ⁿ |θⱼ|
Forces some weights to exactly zero (feature selection)
L2 Regularization (Ridge):
Example
J(θ) = J₀(θ) + (λ/2m) Σⱼ₌₁ⁿ θⱼ²
Shrinks all weights proportionally
Elastic Net:
Example
J(θ) = J₀(θ) + (λ₁/2m) Σⱼ |θⱼ| + (λ₂/2m) Σⱼ θⱼ²
Combines L1 and L2
Evaluation Metrics
Accuracy:
Example
Accuracy = (TP + TN) / (TP + FP + TN + FN)
Precision:
Example
Precision = TP / (TP + FP)
Of predicted positives, how many correct?
Recall (Sensitivity):
Example
Recall = TP / (TP + FN)
Of actual positives, how many detected?
F1-Score:
Example
F1 = 2 * (Precision * Recall) / (Precision + Recall)
Harmonic mean of precision and recall
Specificity:
Example
Specificity = TN / (TN + FP)
Of actual negatives, how many correct?
ROC-AUC:
Example
AUC = ∫₀¹ TPR(FPR⁻¹(x)) dx
Area under ROC curve, [0,1] where 0.5 = random
Regression Metrics
Mean Squared Error (MSE):
Example
MSE = (1/n) Σᵢ₌₁ⁿ (yᵢ - ŷᵢ)²
Average squared difference
Root Mean Squared Error (RMSE):
Example
RMSE = √MSE
Same units as y
Mean Absolute Error (MAE):
Example
MAE = (1/n) Σᵢ₌₁ⁿ |yᵢ - ŷᵢ|
Less sensitive to outliers than MSE
R² Score:
Example
R² = 1 - (SS_res / SS_tot)
SS_res = Σ(yᵢ - ŷᵢ)² (residual sum of squares)
SS_tot = Σ(yᵢ - ȳ)² (total sum of squares)
R² ∈ [0, 1], higher is better (1 = perfect)
Clustering Metrics
Silhouette Score:
Example
s(i) = (b(i) - a(i)) / max(a(i), b(i))
a(i) = avg distance to other points in cluster
b(i) = avg distance to other clusters
s(i) ∈ [-1, 1], higher is better
Davies-Bouldin Index:
DB = (1/k) Σᵢ₌₁ᵏ max(dᵢⱼ)
dᵢⱼ = (σᵢ + σⱼ) / dₙ(cᵢ, cⱼ)
σ = avg distance to cluster center
Lower is better
Inertia (K-Means):
Example
Inertia = Σᵢ₌₁ⁿ ||xᵢ - cᵢ||²
Sum of squared distances to nearest centroid
Probability & Statistics
Bayes' Theorem:
Example
P(A|B) = P(B|A) * P(A) / P(B)
Posterior = (Likelihood * Prior) / Evidence
Normal Distribution (Gaussian):
f(x) = (1 / (σ√(2π))) * exp(-(x - μ)² / (2σ²))
μ = mean, σ = standard deviation
Binomial Distribution:
Example
P(X = k) = C(n,k) * p^k * (1-p)^(n-k)
n = trials, k = successes, p = probability
C(n,k) = n! / (k!(n-k)!)
Conditional Probability:
P(A|B) = P(A ∩ B) / P(B)
Total Probability:
Example
P(A) = Σᵢ P(A|Bᵢ) * P(Bᵢ)
Linear Algebra
Matrix Multiplication:
Example
(AB)ᵢⱼ = Σₖ Aᵢₖ * Bₖⱼ
Transpose:
Example
(Aᵀ)ᵢⱼ = Aⱼᵢ
Inverse:
Example
A⁻¹ = (1/det(A)) * adj(A)
If AA⁻¹ = I (identity matrix)
Determinant (2x2):
Example
det([[a, b], [c, d]]) = ad - bc
Eigenvalues/Eigenvectors:
Example
Av = λv
v = eigenvector, λ = eigenvalue
Norm (L2 / Euclidean):
Example
||x||₂ = √(Σᵢ xᵢ²)
Norm (L1 / Manhattan):
Example
||x||₁ = Σᵢ |xᵢ|
Neural Networks
Activation Functions:
| ReLU | f(x) = max(0, x) |
| Sigmoid | f(x) = 1 / (1 + e^(-x)) |
| Tanh | f(x) = (e^x - e^(-x)) / (e^x + e^(-x)) |
| Softmax | fᵢ(x) = e^(xᵢ) / Σⱼ e^(xⱼ) |
Forward Pass:
z = Wx + b (linear transformation)
a = σ(z) (activation)
Backpropagation (Chain Rule):
Example
∂L/∂w = (∂L/∂a) * (∂a/∂z) * (∂z/∂w)
L = loss, a = activation, z = pre-activation
Weight Update:
Example
w_new = w_old - η * (∂L/∂w)
η = learning rate
Batch Normalization:
x̂ᵢ = (xᵢ - μ_batch) / √(σ²_batch + ε)
Normalize batch, scale with γ, shift with β
Decision Trees & Ensemble
Information Gain (ID3):
Example
Gain(S, A) = Entropy(S) - Σᵥ (|Sᵥ|/|S|) * Entropy(Sᵥ)
A = attribute, S = dataset
Entropy:
Code example
Entropy(S) = -Σᵢ pᵢ * log₂(pᵢ)
pᵢ = proportion of class iGini Impurity:
Example
Gini(S) = 1 - Σᵢ pᵢ²
Summary
Print this and refer back during:
- Implementing algorithms
- Debugging models
- Understanding research papers
- Interview preparation
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Important ML Formulas.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Machine Learning topic.
Search Terms
machine-learning, machine learning, machine, learning, resources, important, formulas, important ml formulas
Related Machine Learning Topics
Continue learning this concept
ResourcesDatasets for ML PracticeCollection of publicly available datasets for practicing machine learning algorithms and building projects.ResourcesMachine Learning GlossaryComprehensive glossary of machine learning terms, definitions, and concepts.ResourcesRecommended ML Books and CoursesCurated list of best machine learning books, courses, and learning resources for all skill levels.ClassificationIntroduction to ClassificationComplete introduction to classification in machine learning covering binary and multi-class problems, decision boundaries, and algorithm selection strategies.ClassificationClassification ProjectsHands-on classification projects covering spam detection, disease prediction, image classification, and customer churn with complete Python implementations.ClassificationDecision TreesMaster decision tree classification with entropy, information gain, Gini impurity, pruning techniques, and complete sklearn implementation guide.