Python Notes
HubHistory of Python - Complete Guide 2026Python Applications - Complete Guide 2026Python Features - Complete Guide 2026What is Python? - Complete Guide 2026Why Learn Python? - Complete Guide 2026Your First Python Program – Hello World & Beyond 2026How to Install Python on Windows, Mac & Linux – Complete Guide 2026Install VS Code for Python – Complete Setup Guide with Extensions 2026Python Environment Setup – Virtual Environments, pip & Project Structure 2026Python IDLE – Complete Guide for Beginners 2026Python CommentsPython Data TypesPython Input and OutputPython OperatorsPython SyntaxPython Strings BasicsPython Type CastingPython VariablesBreak, Continue and Pass in PythonElif Statement in PythonFor Loop in PythonIf-Else Statement in PythonIf Statement in PythonNested If in PythonPattern Programs in PythonWhile Loop in PythonDecorators in PythonFunction Arguments in PythonFunctions Basics in PythonGenerators in PythonLambda Functions in PythonRecursive Functions in PythonReturn Statement in PythonScope of Variables in PythonAbstraction in PythonClasses and Objects in PythonConstructors in PythonEncapsulation in PythonInheritance in PythonMagic Methods (Dunder Methods) in PythonMethod Overriding in PythonPolymorphism in PythonPython Comprehensions – List, Dict, Set & GeneratorPython Dictionaries – Key-Value StorePython Dictionary Methods – Complete ReferencePython List Methods – All Built-in MethodsPython Lists – Complete GuidePython Set Methods – Complete ReferencePython Sets – Unordered Unique CollectionsPython Tuples – Immutable SequencesPython Asynchronous Programming with asyncioPython Context ManagersAdvanced Python DecoratorsAdvanced Python GeneratorsPython Iterators - Advanced GuidePython Memory ManagementPython MultiprocessingPython MultithreadingCRUD Operations in Python DatabasesMySQL with Python (mysql-connector-python)ORM Introduction with SQLAlchemyPostgreSQL with Python (psycopg2)SQLite3 with PythonIntroduction to DjangoDjango Models and ORMDjango REST API with DRFDjango ViewsIntroduction to FlaskFlask RoutingFlask Templates with Jinja2Email Automation with PythonPDF Automation with PythonSelenium Web Automation with PythonTask Scheduler with PythonWeb Scraping with PythonData Analysis with PythonData Cleaning with PythonData Visualization ProjectData Visualization with MatplotlibNumPy IntroductionPandas IntroductionStatistical Visualization with SeabornClassification AlgorithmsClustering AlgorithmsIntroduction to Machine LearningMachine Learning Project: Churn PredictionModel Evaluation and SelectionRegression AlgorithmsSave pipeline (includes preprocessing + model)AI Assistant ProjectCalculator ProjectChatbot ProjectExpense Tracker ProjectFace Detection ProjectPassword Generator ProjectWeather App ProjectPython Coding Interview QuestionsPython OOP Interview Questions and AnswersPython Master CheatsheetPython Interview Questions and AnswersPython Best PracticesCommon Python Errors and SolutionsPython Debugging TechniquesPython Learning Resources
Learn about save pipeline (includes preprocessing + model) in Python Programming with detailed explanations and examples.
⚠️ Common Mistakes
| # | ❌ Mistake | ✅ Correct Way |
|---|---|---|
| 1 | Test data पर fit_transform() use करना — इससे data leakage होता है क्योंकि model ने test data "देख" लिया | Training data पर fit_transform() करो, test data पर सिर्फ transform() use करो |
| 2 | Cross-validation से पहले scaling करना — पूरे dataset पर scaler fit करने से information leak होती है | Pipeline के अंदर scaling रखो ताकि हर fold में separately fit हो |
| 3 | LabelEncoder को features पर use करना — यह ordinal relationship assume करता है (Red=0 < Blue=1 < Green=2) | Nominal features के लिए OneHotEncoder use करो, LabelEncoder सिर्फ target variable के लिए |
| 4 | Imbalanced data पर सिर्फ accuracy देखना — 95% majority class हो तो model सब "majority" predict करके 95% accuracy दे देगा | f1_score, precision, recall, roc_auc_score use करो और class_weight='balanced' set करो |
| 5 | GridSearchCV में बहुत large parameter grid — 5 params × 10 values each = 1,00,000 combinations! | पहले RandomizedSearchCV से approximate range find करो, फिर narrow GridSearchCV use करो |
| 6 | Random state set नहीं करना — हर run पर different results मिलते हैं, reproducibility खत्म | random_state=42 (या कोई fixed number) हमेशा set करो for reproducible experiments |
| 7 | Feature scaling बिना pipeline के — manually scaling करने पर test data leak हो सकता है और code messy बनता है | हमेशा Pipeline या make_pipeline use करो — यह automatically correct order maintain करता है |
✅ Key Takeaways
- 🎯 Scikit-learn का Estimator API consistent है — हर model में
fit(),predict(),score()same तरीके से काम करता है, चाहे Linear Regression हो या Random Forest - 🔄 Pipeline data leakage prevent करता है — preprocessing steps को pipeline में wrap करने से cross-validation और test evaluation दोनों safe रहते हैं
- 📊 ColumnTransformer mixed data handle करता है — numeric columns पर scaling और categorical columns पर encoding एक साथ, एक ही pipeline में
- 🎛️ Hyperparameter tuning systematic हो — RandomizedSearchCV से broad search करो, फिर GridSearchCV से fine-tune करो
- 📈 Cross-validation single train/test split से better है — 5-fold CV model की true performance का more reliable estimate देता है
- ⚖️ Right metric choose करो — Classification में accuracy हमेशा best metric नहीं, imbalanced data पर F1/AUC use करो
- 🧹 Preprocessing matters — StandardScaler (SVM, Logistic Regression के लिए), RobustScaler (outliers हों तो), MinMaxScaler (neural networks के लिए)
- 🔍 Feature importance model interpretability देती है —
feature_importances_औरpermutation_importanceसे समझो कौन से features important हैं - 📉 Learning curves overfitting/underfitting diagnose करती हैं — train-val gap बड़ा = overfitting, दोनों low = underfitting
- 💾
n_jobs=-1parallel processing enable करता है — GridSearchCV और cross_val_score में use करो for faster execution
❓ FAQ
Q1: Pipeline और manually preprocessing करने में क्या difference है? 🤔
Answer: Pipeline automatically ensures कि fit सिर्फ training data पर होता है। Manually करने पर accidentally test data पर fit_transform() call हो सकता है (data leakage)। Pipeline cross-validation और GridSearchCV के साथ seamlessly integrate होता है — हर fold में fresh fit होता है। Production deployment में भी pipeline single object है जो input लेकर output देता है।
Q2: StandardScaler vs MinMaxScaler vs RobustScaler — कब कौन सा use करें? 📐
Answer:
- StandardScaler: Default choice, SVM और Logistic Regression के लिए best (assumes Gaussian distribution)
- MinMaxScaler: जब features bounded range में चाहिए [0,1] — Neural Networks और image data के लिए
- RobustScaler: जब data में outliers हों — median और IQR use करता है, outliers से affect नहीं होता
Q3: GridSearchCV और RandomizedSearchCV में क्या choose करें? 🎯
Answer: GridSearchCV सब combinations try करता है — small parameter space (< 100 combinations) के लिए best। RandomizedSearchCV random sampling करता है — large search space में faster है, 50-100 iterations में often good results मिल जाते हैं। Best strategy: पहले RandomizedSearchCV से broad area identify करो, फिर GridSearchCV से fine-tune करो।
Q4: Cross-validation में cv=5 ही क्यों use करते हैं? 🔢
Answer: 5-fold एक good balance है bias-variance tradeoff में। कम folds (cv=3) = higher bias, ज़्यादा folds (cv=10) = higher variance और slower। Small datasets पर cv=10 या LeaveOneOut better है। Large datasets पर cv=3 भी sufficient है। StratifiedKFold classification में prefer करो क्योंकि यह class proportions maintain करता है।
Q5: fit_transform() और fit() + transform() separately call करने में difference? 🔧
Answer: fit_transform() = fit() + transform() एक step में — computationally optimized हो सकता है (कुछ transformers जैसे PCA internally ज़्यादा efficient implementation use करते हैं)। Rule: Training data पर fit_transform(), test data पर सिर्फ transform()। Pipeline use करो तो यह automatically handle होता है।
Q6: OneHotEncoder में drop='first' क्यों use करते हैं? 🎭
Answer: Dummy variable trap avoid करने के लिए! अगर 3 categories हैं (Red, Blue, Green) तो 2 columns sufficient हैं — तीसरी column linearly dependent है (अगर Red=0, Blue=0 तो Green=1 ही होगा)। drop='first' multicollinearity reduce करता है जो Linear Regression और Logistic Regression की performance affect करती है। Tree-based models को इससे फ़र्क़ नहीं पड़ता।
Q7: Model save/load कैसे करें production में? 💾
Answer:
python example
import joblib
# Save pipeline (includes preprocessing + model)
joblib.dump(pipeline, 'model_pipeline.joblib')
# Load and predict
loaded_pipeline = joblib.load('model_pipeline.joblib')
predictions = loaded_pipeline.predict(new_data)Pipeline save करो (not just model) ताकि preprocessing steps भी included रहें। pickle भी काम करता है लेकिन joblib NumPy arrays के साथ faster है।
Q8: n_jobs=-1 कब use करना चाहिए और कब नहीं? ⚡
Answer: n_jobs=-1 सारे CPU cores use करता है — use करो GridSearchCV, cross_val_score, RandomForest training में। Avoid करो जब: (1) dataset बहुत small है (overhead > benefit), (2) already parallel environment में run कर रहे हो (Jupyter + multiprocessing conflict), (3) memory intensive operations हों (each core full copy लेता है)। Default n_jobs=1 safe choice है small data पर।
🎯 Interview Questions
Q1: Scikit-learn का Estimator API explain करो। fit(), predict(), और transform() में क्या difference है?
Answer: Scikit-learn में हर algorithm एक Estimator object है जो consistent interface follow करता है:
fit(X, y): Training data से model parameters learn करता है (coefficients, tree splits, etc.)predict(X): Learned parameters use करके new data पर predictions generate करता हैtransform(X): Data को transform करता है (scaling, encoding) — Transformers (StandardScaler, PCA) में होता हैfit_transform(X): fit + transform एक step में — computationally optimizedscore(X, y): Model performance evaluate करता है (R² for regression, accuracy for classification)get_params()/set_params(): Hyperparameters access/modify करता है
यह consistency मतलब आप कोई भी model swap कर सकते हो बिना code change किए — बस estimator object change करो।
Q2: Data leakage क्या है और Pipeline इसे कैसे prevent करता है?
Answer: Data leakage = जब model training में ऐसी information use हो जो real-world prediction time पर available नहीं होगी।
Example: पूरे dataset पर StandardScaler fit करना — test data का mean/std model ने "देख" लिया।
Pipeline solution: Pipeline ensure करता है कि cross-validation की हर fold में:
- Scaler सिर्फ training fold पर fit होता है
- Validation fold पर सिर्फ transform होता है
- कोई information leak नहीं होती
बिना Pipeline: scaler.fit_transform(X) → train_test_split → LEAKED! With Pipeline: split → pipeline.fit(X_train) → pipeline.predict(X_test) → SAFE! ✅
Q3: GridSearchCV internally कैसे काम करता है? Explain with example.
Answer: GridSearchCV exhaustive search करता है सभी parameter combinations पर:
- Parameter grid define करो:
{'C': [0.1, 1, 10], 'kernel': ['rbf', 'linear']}= 6 combinations - हर combination के लिए k-fold cross-validation run करो (cv=5 = 5 folds per combination)
- Total fits = 6 × 5 = 30 model trainings
- हर combination का mean CV score calculate करो
- Best score वाली combination select करो
- Final model best params के साथ पूरे training data पर refit करो (
refit=Truedefault)
Result: grid_search.best_params_, grid_search.best_score_, grid_search.best_estimator_ (already fitted)
Q4: Stratified K-Fold और regular K-Fold में क्या difference है? कब कौन सा use करें?
Answer:
- KFold: Data randomly k parts में split करता है — class distribution maintain नहीं करता
- StratifiedKFold: हर fold में original dataset जैसी class proportions maintain करता है
Example: 90% class-0, 10% class-1 → Regular KFold में कोई fold सिर्फ class-0 हो सकता है → misleading results!
Use StratifiedKFold when: Classification problems, imbalanced datasets Use regular KFold when: Regression problems (no classes to stratify)
Q5: Feature scaling क्यों ज़रूरी है? कौन से algorithms को scaling चाहिए और कौन से को नहीं?
Answer:
Scaling ज़रूरी है (distance/gradient based):
- SVM, KNN (distance calculations affected)
- Logistic Regression, Linear Regression (gradient descent convergence)
- PCA (variance-based, larger features dominate)
- Neural Networks (gradient flow)
Scaling NOT needed (tree-based, splits पर based):
- Decision Trees, Random Forest, XGBoost, LightGBM
- Naive Bayes (probability-based)
Reason: अगर Age [0-100] और Salary [20000-200000] हो, तो distance-based algorithms salary को ज़्यादा weight देंगे सिर्फ scale की वजह से — actual importance नहीं।
Q6: ColumnTransformer का real-world use case explain करो।
Answer: Real datasets में mixed types होते हैं — numeric + categorical + text। ColumnTransformer different pipelines apply करता है different columns पर:
Example
preprocessor = ColumnTransformer([
('num', Pipeline([('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler())]), numeric_cols),
('cat', Pipeline([('imputer', SimpleImputer(strategy='most_frequent')),
('encoder', OneHotEncoder())]), categorical_cols),
('text', TfidfVectorizer(), 'description_col'), # text column
])
Key benefit: सब एक Pipeline में — production में single .predict() call input लेकर output देता है। No manual preprocessing needed at inference time।
Q7: Overfitting detect करने के लिए कौन सी techniques use करोगे?
Answer:
- Train vs Test score comparison: Train accuracy 99%, Test accuracy 75% = overfitting
- Learning curves: Train score high, validation score low with gap = overfitting
- Cross-validation: High variance across folds suggests overfitting
- Regularization impact: अगर regularization (C, alpha) बढ़ाने से test score improve हो = overfitting था
Solutions:
- Regularization बढ़ाओ (lower C in SVM, higher alpha in Ridge)
- Max_depth limit करो (trees)
- More training data collect करो
- Feature selection (reduce dimensionality)
- Dropout / early stopping (neural networks)
Q8: RandomizedSearchCV GridSearchCV से better कब होता है?
Answer:
- GridSearchCV: Small, discrete parameter space — 3 params × 4 values = 64 combinations, feasible
- RandomizedSearchCV: Large/continuous space — 6 params × continuous distributions = infinite combinations
RandomizedSearchCV advantages:
- Computational budget fix कर सकते हो (
n_iter=50) - Continuous distributions sample कर सकते हो (
uniform(0.01, 10)) - Research shows random search often finds good params faster than grid (Bergstra & Bengio, 2012)
- Important parameters पर ज़्यादा values try होते हैं automatically
Best practice: RandomizedSearchCV (broad, n_iter=100) → identify promising region → GridSearchCV (narrow, focused)
Q9: cross_val_score vs cross_validate — कब कौन सा use करें?
Answer:
cross_val_score: Single metric return करता है — simple evaluation के लिएcross_validate: Multiple metrics, train scores, fit/score times — detailed analysis के लिए
Example
# Simple — single metric
scores = cross_val_score(model, X, y, cv=5, scoring='accuracy')
# Detailed — multiple metrics + timing
results = cross_validate(model, X, y, cv=5,
scoring=['accuracy', 'f1', 'roc_auc'],
return_train_score=True)
# results['test_accuracy'], results['train_accuracy'], results['fit_time']
Use cross_validate जब: multiple metrics compare करने हों, overfitting check करना हो (train vs test), या training time measure करना हो।
Q10: Production में ML model deploy करने के लिए scikit-learn best practices क्या हैं?
Answer:
- Pipeline save करो (not just model) — preprocessing included रहे
joblib.dump()use करो for serialization (better than pickle for numpy arrays)- Version pin करो — sklearn version mismatch से model load fail हो सकता है
- Input validation — new data का shape/type check करो before prediction
- Monitoring — prediction distribution track करो, data drift detect करो
Example
import joblib
from sklearn import __version__
# Save with metadata
model_artifact = {
'pipeline': full_pipeline,
'sklearn_version': __version__,
'feature_names': feature_names,
'training_date': '2026-06-12'
}
joblib.dump(model_artifact, 'model_v1.joblib')
# Load and validate
artifact = joblib.load('model_v1.joblib')
assert artifact['sklearn_version'] == __version__, "Version mismatch!"
pipeline = artifact['pipeline']
*Next Lesson: Regression →* title: "Scikit-Learn Deep Dive", description: "Master scikit-learn, Python's most popular ML library. Learn the estimator API, preprocessing, pipelines, model selection, hyperparameter tuning with GridSearchCV, and cross-validation techniques.", keywords: [ "scikit-learn", "sklearn tutorial", "scikit-learn api", "sklearn pipeline", "sklearn preprocessing", "sklearn StandardScaler", "sklearn GridSearchCV", "sklearn cross_val_score", "sklearn train_test_split", "sklearn metrics", "sklearn model selection", "sklearn hyperparameter tuning", "sklearn RandomizedSearchCV", "sklearn Pipeline", "sklearn ColumnTransformer", "sklearn OneHotEncoder", "sklearn LabelEncoder", "sklearn MinMaxScaler", "sklearn RobustScaler", "sklearn feature selection", "sklearn FeatureUnion", "sklearn Imputer", "sklearn learning_curve", "sklearn confusion_matrix", "sklearn roc_auc_score", "sklearn classification_report", "sklearn cross_validate", "sklearn make_pipeline", "sklearn estimator", "sklearn transformer", "sklearn fit predict transform" ], publishedAt: "2026-06-12", updatedAt: "2026-06-12", category: "Python", difficulty: "Intermediate", author: "WoHoTech" };
Scikit-learn is Python's most popular machine learning library, providing a consistent API for hundreds of algorithms, preprocessing tools, and model evaluation utilities.
Installation
bash example
pip install scikit-learn pandas numpy matplotlibpython example
import sklearn
print(sklearn.__version__) # 1.4.x or laterThe Estimator API
Everything in scikit-learn follows the same interface:
Example
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
# Generate sample data
X, y = make_regression(n_samples=200, n_features=5, noise=10, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create estimator
model = LinearRegression()
# .fit() — Learn from training data
model.fit(X_train, y_train)
# .predict() — Make predictions
y_pred = model.predict(X_test)
# .score() — Returns R² for regression, accuracy for classification
r2 = model.score(X_test, y_test)
print(f"R² Score: {r2:.4f}")
# Access learned parameters
print(f"Coefficients: {model.coef_[:3]}")
print(f"Intercept: {model.intercept_:.4f}")
# get_params() — See hyperparameters
print(f"Parameters: {model.get_params()}")
# set_params() — Change hyperparameters
model.set_params(fit_intercept=False)
Data Preprocessing
StandardScaler
Example
from sklearn.preprocessing import StandardScaler
import numpy as np
X = np.array([[1, 100], [2, 200], [3, 150], [4, 300], [5, 250]])
scaler = StandardScaler()
# IMPORTANT: fit on training data only, then transform both
X_train = X[:4]
X_test = X[4:]
# Fit on training set (learns mean and std)
scaler.fit(X_train)
print(f"Mean: {scaler.mean_}")
print(f"Std: {scaler.scale_}")
# Transform
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test) # Use TRAINING stats on test
# fit_transform = fit + transform in one step (use only on training data)
X_train_scaled = scaler.fit_transform(X_train)
print(f"\nScaled X_train:\n{X_train_scaled.round(3)}")
# Each feature now has mean ≈ 0 and std ≈ 1
MinMaxScaler and RobustScaler
Example
from sklearn.preprocessing import MinMaxScaler, RobustScaler
data = np.array([[1.0], [5.0], [10.0], [15.0], [1000.0]]) # Has outlier
# MinMaxScaler — scales to [0, 1] range (sensitive to outliers)
min_max = MinMaxScaler()
print("MinMax:", min_max.fit_transform(data).flatten())
# [0. 0.004 0.009 0.014 1. ] — outlier dominates!
# RobustScaler — uses median and IQR (robust to outliers)
robust = RobustScaler()
print("Robust:", robust.fit_transform(data).flatten())
# More balanced, outlier less influential
Encoding Categorical Variables
Example
from sklearn.preprocessing import LabelEncoder, OrdinalEncoder, OneHotEncoder
import pandas as pd
import numpy as np
# Sample data
data = pd.DataFrame({
'color': ['Red', 'Blue', 'Green', 'Red', 'Blue'],
'size': ['S', 'M', 'L', 'XL', 'M'],
'quality': ['Low', 'Medium', 'High', 'Medium', 'High'],
})
# LabelEncoder — for target variable only (ordinal numbers, loses info)
le = LabelEncoder()
data['color_encoded'] = le.fit_transform(data['color'])
print(f"Label encoded: {data[['color', 'color_encoded']].head()}")
# OrdinalEncoder — when there's a natural order
size_order = [['S', 'M', 'L', 'XL']]
oe = OrdinalEncoder(categories=size_order)
data['size_encoded'] = oe.fit_transform(data[['size']])
print(f"\nOrdinal encoded sizes: {data['size_encoded'].values}")
# OneHotEncoder — preferred for nominal (unordered) categories
ohe = OneHotEncoder(sparse_output=False, drop='first') # drop first avoids dummy trap
color_ohe = ohe.fit_transform(data[['color']])
ohe_df = pd.DataFrame(color_ohe, columns=ohe.get_feature_names_out(['color']))
print(f"\nOne-hot encoded:\n{ohe_df}")
# Pandas get_dummies — quick alternative
color_dummies = pd.get_dummies(data['color'], prefix='color', drop_first=True)
print(f"\nPandas dummies:\n{color_dummies}")
Handling Missing Values
Example
from sklearn.impute import SimpleImputer, KNNImputer
import numpy as np
X = np.array([[1, 2, np.nan],
[3, np.nan, 6],
[np.nan, 8, 9],
[4, 5, 6]])
# Strategy: mean, median, most_frequent, constant
mean_imputer = SimpleImputer(strategy='mean')
print("Mean imputed:\n", mean_imputer.fit_transform(X))
median_imputer = SimpleImputer(strategy='median')
print("\nMedian imputed:\n", median_imputer.fit_transform(X))
const_imputer = SimpleImputer(strategy='constant', fill_value=0)
print("\nConstant imputed:\n", const_imputer.fit_transform(X))
# KNN Imputer — fills based on k nearest neighbors
knn_imputer = KNNImputer(n_neighbors=2)
print("\nKNN imputed:\n", knn_imputer.fit_transform(X))
Pipelines
Pipelines chain multiple steps together:
Example
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
# Load data
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Pipeline — no need to manually scale train/test separately
pipeline = Pipeline([
('scaler', StandardScaler()),
('classifier', LogisticRegression(max_iter=1000, random_state=42))
])
# Alternative — make_pipeline (auto-names steps)
pipeline = make_pipeline(
StandardScaler(),
LogisticRegression(max_iter=1000, random_state=42)
)
# Train
pipeline.fit(X_train, y_train)
# Evaluate
print(f"Train accuracy: {pipeline.score(X_train, y_train):.4f}")
print(f"Test accuracy: {pipeline.score(X_test, y_test):.4f}")
# Access steps
print(f"\nScaler mean (first 3): {pipeline.named_steps['standardscaler'].mean_[:3]}")
print(f"Model classes: {pipeline.named_steps['logisticregression'].classes_}")
# Predict with full pipeline (auto-scales new data)
new_data = X_test[:3]
predictions = pipeline.predict(new_data)
probabilities = pipeline.predict_proba(new_data)
print(f"\nPredictions: {predictions}")
print(f"Probabilities: {probabilities[:, 1].round(3)}")
ColumnTransformer — Different Transforms for Different Columns
Example
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
import pandas as pd
import numpy as np
# Mixed dataset (numeric + categorical)
df = pd.DataFrame({
'age': [25, 30, np.nan, 45, 35],
'salary': [50000, 75000, 60000, 90000, np.nan],
'city': ['NYC', 'LA', 'NYC', 'Chicago', 'LA'],
'education': ['Bachelor', 'Master', 'PhD', 'Bachelor', 'Master'],
'employed': [1, 0, 1, 1, 0],
})
X = df.drop('employed', axis=1)
y = df['employed']
numeric_features = ['age', 'salary']
categorical_features = ['city', 'education']
# Numeric pipeline: impute + scale
numeric_pipeline = Pipeline([
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler())
])
# Categorical pipeline: impute + encode
categorical_pipeline = Pipeline([
('imputer', SimpleImputer(strategy='most_frequent')),
('encoder', OneHotEncoder(handle_unknown='ignore', sparse_output=False))
])
# Combine with ColumnTransformer
preprocessor = ColumnTransformer([
('num', numeric_pipeline, numeric_features),
('cat', categorical_pipeline, categorical_features)
])
# Full pipeline
full_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', LogisticRegression())
])
# Train and evaluate
from sklearn.model_selection import cross_val_score
scores = cross_val_score(full_pipeline, X, y, cv=3, scoring='accuracy')
print(f"CV Accuracy: {scores.mean():.3f} ± {scores.std():.3f}")
Cross-Validation
Example
from sklearn.model_selection import (
cross_val_score, cross_validate,
StratifiedKFold, KFold,
LeaveOneOut
)
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
import numpy as np
X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(n_estimators=100, random_state=42)
# Basic k-fold cross-validation
cv_scores = cross_val_score(model, X, y, cv=5, scoring='accuracy')
print(f"5-fold CV: {cv_scores}")
print(f"Mean ± Std: {cv_scores.mean():.4f} ± {cv_scores.std():.4f}")
# Stratified k-fold (preserves class proportions — use for classification)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores_stratified = cross_val_score(model, X, y, cv=skf, scoring='accuracy')
print(f"\nStratified CV: {scores_stratified.mean():.4f}")
# cross_validate — multiple metrics at once
results = cross_validate(
model, X, y, cv=5,
scoring=['accuracy', 'f1_macro', 'roc_auc_ovr'],
return_train_score=True
)
print(f"\nTrain accuracy: {results['train_accuracy'].mean():.4f}")
print(f"Test accuracy: {results['test_accuracy'].mean():.4f}")
print(f"Test F1 macro: {results['test_f1_macro'].mean():.4f}")
Hyperparameter Tuning
GridSearchCV
Example
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipeline = Pipeline([
('scaler', StandardScaler()),
('svm', SVC(random_state=42))
])
# Parameter grid (use __ to access pipeline step parameters)
param_grid = {
'svm__C': [0.1, 1, 10, 100],
'svm__kernel': ['rbf', 'linear'],
'svm__gamma': ['scale', 'auto'],
}
grid_search = GridSearchCV(
pipeline, param_grid,
cv=5, scoring='accuracy',
n_jobs=-1, # Use all CPU cores
verbose=1,
return_train_score=True
)
grid_search.fit(X, y)
print(f"Best parameters: {grid_search.best_params_}")
print(f"Best CV score: {grid_search.best_score_:.4f}")
# Best estimator (already fitted)
best_model = grid_search.best_estimator_
# Results DataFrame
results = pd.DataFrame(grid_search.cv_results_)
top5 = results.sort_values('rank_test_score').head(5)[
['params', 'mean_test_score', 'std_test_score', 'rank_test_score']
]
print(f"\nTop 5 configurations:\n{top5}")
RandomizedSearchCV
Example
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier
from scipy.stats import randint, uniform
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
# Distribution-based parameter space
param_dist = {
'n_estimators': randint(50, 500),
'max_depth': [None, 10, 20, 30],
'min_samples_split': randint(2, 20),
'min_samples_leaf': randint(1, 10),
'max_features': ['sqrt', 'log2', None],
'bootstrap': [True, False],
}
random_search = RandomizedSearchCV(
RandomForestClassifier(random_state=42),
param_distributions=param_dist,
n_iter=50, # Try 50 random combinations (faster than grid)
cv=5,
scoring='accuracy',
n_jobs=-1,
random_state=42,
verbose=1
)
random_search.fit(X, y)
print(f"Best params: {random_search.best_params_}")
print(f"Best score: {random_search.best_score_:.4f}")
Model Evaluation Metrics
Example
from sklearn.metrics import (
accuracy_score, precision_score, recall_score, f1_score,
confusion_matrix, classification_report,
roc_auc_score, roc_curve, average_precision_score,
mean_squared_error, mean_absolute_error, r2_score
)
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
# Classification metrics example
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(scaler.fit_transform(X_train), y_train)
y_pred = clf.predict(scaler.transform(X_test))
y_prob = clf.predict_proba(scaler.transform(X_test))[:, 1]
# Core metrics
print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
print(f"Precision: {precision_score(y_test, y_pred):.4f}")
print(f"Recall: {recall_score(y_test, y_pred):.4f}")
print(f"F1 Score: {f1_score(y_test, y_pred):.4f}")
print(f"ROC AUC: {roc_auc_score(y_test, y_prob):.4f}")
# Full report
print(f"\n{classification_report(y_test, y_pred, target_names=['Malignant', 'Benign'])}")
# Confusion matrix
cm = confusion_matrix(y_test, y_pred)
plt.figure(figsize=(6, 5))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
xticklabels=['Malignant', 'Benign'],
yticklabels=['Malignant', 'Benign'])
plt.title('Confusion Matrix')
plt.ylabel('True Label')
plt.xlabel('Predicted Label')
plt.tight_layout()
plt.show()
# ROC Curve
fpr, tpr, thresholds = roc_curve(y_test, y_prob)
auc = roc_auc_score(y_test, y_prob)
plt.figure(figsize=(7, 5))
plt.plot(fpr, tpr, 'b-', linewidth=2, label=f'ROC curve (AUC = {auc:.3f})')
plt.plot([0, 1], [0, 1], 'r--', linewidth=1, label='Random classifier')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()
Feature Importance
python example
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
import pandas as pd
import matplotlib.pyplot as plt
X, y = load_breast_cancer(return_X_y=True)
feature_names = load_breast_cancer().feature_names
clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X, y)
# Built-in feature importance (impurity-based)
importances = pd.Series(clf.feature_importances_, index=feature_names)
top_features = importances.nlargest(10)
plt.figure(figsize=(10, 6))
top_features.sort_values().plot(kind='barh', color='steelblue', alpha=0.8)
plt.title('Top 10 Feature Importances (Random Forest)')
plt.xlabel('Importance Score')
plt.tight_layout()
plt.show()
print("Top 5 features:")
print(top_features.head())Learning Curves
python example
from sklearn.model_selection import learning_curve
import matplotlib.pyplot as plt
import numpy as np
def plot_learning_curve(estimator, X, y, title='Learning Curve', cv=5):
train_sizes, train_scores, val_scores = learning_curve(
estimator, X, y, cv=cv, n_jobs=-1,
train_sizes=np.linspace(0.1, 1.0, 10),
scoring='accuracy'
)
train_mean = train_scores.mean(axis=1)
train_std = train_scores.std(axis=1)
val_mean = val_scores.mean(axis=1)
val_std = val_scores.std(axis=1)
plt.figure(figsize=(10, 6))
plt.fill_between(train_sizes, train_mean - train_std, train_mean + train_std, alpha=0.2, color='blue')
plt.fill_between(train_sizes, val_mean - val_std, val_mean + val_std, alpha=0.2, color='orange')
plt.plot(train_sizes, train_mean, 'b-o', label='Training score', linewidth=2)
plt.plot(train_sizes, val_mean, 'o-', color='orange', label='Validation score', linewidth=2)
plt.xlabel('Training Set Size')
plt.ylabel('Accuracy Score')
plt.title(title)
plt.legend(loc='lower right')
plt.grid(True, alpha=0.3)
plt.ylim(0, 1.05)
plt.tight_layout()
plt.show()
from sklearn.ensemble import RandomForestClassifier
X, y = load_breast_cancer(return_X_y=True)
plot_learning_curve(RandomForestClassifier(n_estimators=100, random_state=42), X, y,
title='Random Forest Learning Curve')Summary
In this lesson, you learned:
- ✅ Scikit-learn's consistent estimator API
- ✅ Preprocessing: StandardScaler, MinMaxScaler, RobustScaler
- ✅ Encoding: LabelEncoder, OrdinalEncoder, OneHotEncoder
- ✅ Missing value imputation (SimpleImputer, KNNImputer)
- ✅ Building Pipelines to prevent data leakage
- ✅ ColumnTransformer for mixed-type datasets
- ✅ Cross-validation strategies (k-fold, stratified)
- ✅ Hyperparameter tuning (GridSearchCV, RandomizedSearchCV)
- ✅ Evaluation metrics (accuracy, F1, AUC, confusion matrix)
- ✅ Feature importance and learning curves
*Next Lesson: Regression →*
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Save pipeline (includes preprocessing + model).
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, machine, learning, scikit
Related Python Master Course Topics
Continue learning this concept
Machine LearningClassification AlgorithmsLearn classification algorithms in Python — Logistic Regression, Decision Trees, Random Forest, SVM, KNN, Naive Bayes, and Gradient Boosting, with metrics, confusion matrices, ROC curves, and class imbalance handling.Machine LearningClustering AlgorithmsLearn unsupervised clustering algorithms in Python — K-Means, Hierarchical Clustering, DBSCAN, and Gaussian Mixture Models, with evaluation metrics, the elbow method, and customer segmentation project.Machine LearningIntroduction to Machine LearningLearn the fundamentals of machine learning — what it is, types of ML (supervised, unsupervised, reinforcement), key concepts, the ML workflow, and how to set up your Python ML environment.Machine LearningMachine Learning Project: Churn PredictionBuild a complete end-to-end machine learning project — predict customer churn using real-world techniques: EDA, feature engineering, model training, evaluation, hyperparameter tuning, and deployment-ready pipeline.Machine LearningModel Evaluation and SelectionMaster ML model evaluation — cross-validation strategies, evaluation metrics for regression and classification, overfitting detection, hyperparameter tuning, model selection best practices, and model persistence with joblib.Machine LearningRegression AlgorithmsMaster regression algorithms in Python — Linear Regression, Ridge, Lasso, ElasticNet, Polynomial Regression, Decision Tree Regression, and Random Forest Regression with practical examples and evaluation.