Python Notes
HubHistory of Python - Complete Guide 2026Python Applications - Complete Guide 2026Python Features - Complete Guide 2026What is Python? - Complete Guide 2026Why Learn Python? - Complete Guide 2026Your First Python Program – Hello World & Beyond 2026How to Install Python on Windows, Mac & Linux – Complete Guide 2026Install VS Code for Python – Complete Setup Guide with Extensions 2026Python Environment Setup – Virtual Environments, pip & Project Structure 2026Python IDLE – Complete Guide for Beginners 2026Python CommentsPython Data TypesPython Input and OutputPython OperatorsPython SyntaxPython Strings BasicsPython Type CastingPython VariablesBreak, Continue and Pass in PythonElif Statement in PythonFor Loop in PythonIf-Else Statement in PythonIf Statement in PythonNested If in PythonPattern Programs in PythonWhile Loop in PythonDecorators in PythonFunction Arguments in PythonFunctions Basics in PythonGenerators in PythonLambda Functions in PythonRecursive Functions in PythonReturn Statement in PythonScope of Variables in PythonAbstraction in PythonClasses and Objects in PythonConstructors in PythonEncapsulation in PythonInheritance in PythonMagic Methods (Dunder Methods) in PythonMethod Overriding in PythonPolymorphism in PythonPython Comprehensions – List, Dict, Set & GeneratorPython Dictionaries – Key-Value StorePython Dictionary Methods – Complete ReferencePython List Methods – All Built-in MethodsPython Lists – Complete GuidePython Set Methods – Complete ReferencePython Sets – Unordered Unique CollectionsPython Tuples – Immutable SequencesPython Asynchronous Programming with asyncioPython Context ManagersAdvanced Python DecoratorsAdvanced Python GeneratorsPython Iterators - Advanced GuidePython Memory ManagementPython MultiprocessingPython MultithreadingCRUD Operations in Python DatabasesMySQL with Python (mysql-connector-python)ORM Introduction with SQLAlchemyPostgreSQL with Python (psycopg2)SQLite3 with PythonIntroduction to DjangoDjango Models and ORMDjango REST API with DRFDjango ViewsIntroduction to FlaskFlask RoutingFlask Templates with Jinja2Email Automation with PythonPDF Automation with PythonSelenium Web Automation with PythonTask Scheduler with PythonWeb Scraping with PythonData Analysis with PythonData Cleaning with PythonData Visualization ProjectData Visualization with MatplotlibNumPy IntroductionPandas IntroductionStatistical Visualization with SeabornClassification AlgorithmsClustering AlgorithmsIntroduction to Machine LearningMachine Learning Project: Churn PredictionModel Evaluation and SelectionRegression AlgorithmsSave pipeline (includes preprocessing + model)AI Assistant ProjectCalculator ProjectChatbot ProjectExpense Tracker ProjectFace Detection ProjectPassword Generator ProjectWeather App ProjectPython Coding Interview QuestionsPython OOP Interview Questions and AnswersPython Master CheatsheetPython Interview Questions and AnswersPython Best PracticesCommon Python Errors and SolutionsPython Debugging TechniquesPython Learning Resources
Master regression algorithms in Python — Linear Regression, Ridge, Lasso, ElasticNet, Polynomial Regression, Decision Tree Regression, and Random Forest Regression with practical examples and evaluation.
Regression is a type of supervised learning where we predict a continuous numerical value. This lesson covers the major regression algorithms and how to select and evaluate them.
Setup
bash example
pip install scikit-learn pandas numpy matplotlib seaborn xgboostpython example
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_scoreDataset: House Price Prediction
python example
from sklearn.datasets import fetch_california_housing
import pandas as pd
import numpy as np
# Load California Housing dataset
housing = fetch_california_housing()
X = pd.DataFrame(housing.data, columns=housing.feature_names)
y = pd.Series(housing.target, name='median_house_value') * 100000 # In dollars
print(f"Shape: {X.shape}")
print(f"Features: {X.columns.tolist()}")
print(f"Target: ${y.mean():,.0f} average, range ${y.min():,.0f} to ${y.max():,.0f}")
print(f"\n{X.describe().round(2)}")Evaluation Helper
python example
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np
def evaluate_regression(y_true, y_pred, model_name='Model'):
"""Print comprehensive regression evaluation metrics."""
rmse = np.sqrt(mean_squared_error(y_true, y_pred))
mae = mean_absolute_error(y_true, y_pred)
r2 = r2_score(y_true, y_pred)
mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
print(f"\n{'='*40}")
print(f"Model: {model_name}")
print(f"{'='*40}")
print(f" RMSE: ${rmse:>12,.0f}")
print(f" MAE: ${mae:>12,.0f}")
print(f" R²: {r2:>13.4f}")
print(f" MAPE: {mape:>12.2f}%")
return {'rmse': rmse, 'mae': mae, 'r2': r2, 'mape': mape}
# Usage
y_dummy = np.random.normal(250000, 50000, 100)
y_pred_dummy = y_dummy + np.random.normal(0, 30000, 100)
evaluate_regression(y_dummy, y_pred_dummy, 'Dummy Test')Linear Regression
python example
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
# Load and prepare
X, y = fetch_california_housing(return_X_y=True)
y = y * 100000 # Scale to dollars
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Scale features
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
# Train
lr = LinearRegression()
lr.fit(X_train_s, y_train)
# Evaluate
y_pred_lr = lr.predict(X_test_s)
metrics_lr = evaluate_regression(y_test, y_pred_lr, 'Linear Regression')
# Coefficients
feature_names = fetch_california_housing().feature_names
coef_df = pd.DataFrame({
'feature': feature_names,
'coefficient': lr.coef_
}).sort_values('coefficient', key=abs, ascending=False)
print(f"\nFeature Coefficients:\n{coef_df}")
print(f"Intercept: ${lr.intercept_:,.0f}")
# Assumptions check
residuals = y_test - y_pred_lr
print(f"\nResiduals: mean={residuals.mean():.0f}, std={residuals.std():,.0f}")Ridge Regression (L2 Regularization)
Example
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV
# Ridge adds L2 penalty: Loss = MSE + alpha * sum(coef²)
# Shrinks large coefficients toward zero — prevents overfitting
ridge = Ridge(alpha=1.0)
ridge.fit(X_train_s, y_train)
y_pred_ridge = ridge.predict(X_test_s)
metrics_ridge = evaluate_regression(y_test, y_pred_ridge, 'Ridge Regression')
# Tune alpha with GridSearch
alphas = [0.01, 0.1, 1, 10, 100, 1000]
ridge_cv = GridSearchCV(Ridge(), {'alpha': alphas}, cv=5, scoring='r2', n_jobs=-1)
ridge_cv.fit(X_train_s, y_train)
print(f"\nBest Ridge alpha: {ridge_cv.best_params_['alpha']}")
print(f"Best CV R²: {ridge_cv.best_score_:.4f}")
# Compare coefficients: Linear vs Ridge
coef_comparison = pd.DataFrame({
'feature': feature_names,
'linear': lr.coef_,
'ridge': ridge.coef_,
})
print(f"\nCoefficient comparison (Linear vs Ridge):\n{coef_comparison}")
Lasso Regression (L1 Regularization)
python example
from sklearn.linear_model import Lasso, LassoCV
# Lasso adds L1 penalty: Loss = MSE + alpha * sum(|coef|)
# Can drive some coefficients to EXACTLY zero — built-in feature selection!
lasso = Lasso(alpha=100, max_iter=10000)
lasso.fit(X_train_s, y_train)
y_pred_lasso = lasso.predict(X_test_s)
metrics_lasso = evaluate_regression(y_test, y_pred_lasso, 'Lasso Regression')
# See which features were eliminated
lasso_coefs = pd.Series(lasso.coef_, index=feature_names)
print(f"\nLasso Coefficients (zero = eliminated):\n{lasso_coefs}")
print(f"Non-zero features: {(lasso_coefs != 0).sum()}")
print(f"Eliminated features: {(lasso_coefs == 0).sum()}")
# LassoCV — automatic alpha selection
lasso_cv = LassoCV(cv=5, random_state=42, max_iter=10000)
lasso_cv.fit(X_train_s, y_train)
print(f"\nBest Lasso alpha: {lasso_cv.alpha_:.4f}")ElasticNet (L1 + L2)
Example
from sklearn.linear_model import ElasticNet, ElasticNetCV
# ElasticNet = combination of Lasso and Ridge
# l1_ratio = 1 → Lasso, l1_ratio = 0 → Ridge
en = ElasticNet(alpha=100, l1_ratio=0.5, max_iter=10000)
en.fit(X_train_s, y_train)
y_pred_en = en.predict(X_test_s)
metrics_en = evaluate_regression(y_test, y_pred_en, 'ElasticNet')
# Auto-tune with ElasticNetCV
en_cv = ElasticNetCV(l1_ratio=[0.1, 0.5, 0.9, 1.0], cv=5, random_state=42)
en_cv.fit(X_train_s, y_train)
print(f"Best l1_ratio: {en_cv.l1_ratio_}, Best alpha: {en_cv.alpha_:.4f}")
Polynomial Regression
Example
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from sklearn.linear_model import Ridge
# Simple polynomial regression example
np.random.seed(42)
X_1d = np.sort(np.random.uniform(0, 10, 100)).reshape(-1, 1)
y_1d = 2 * X_1d.ravel()**2 - 3 * X_1d.ravel() + 5 + np.random.normal(0, 5, 100)
X_1d_train, X_1d_test, y_1d_train, y_1d_test = train_test_split(X_1d, y_1d, test_size=0.2)
# Compare degrees
fig, axes = plt.subplots(1, 3, figsize=(15, 4))
for degree, ax in zip([1, 2, 5], axes):
poly_pipe = Pipeline([
('poly', PolynomialFeatures(degree=degree)),
('scaler', StandardScaler()),
('model', Ridge(alpha=1.0))
])
poly_pipe.fit(X_1d_train, y_1d_train)
train_r2 = poly_pipe.score(X_1d_train, y_1d_train)
test_r2 = poly_pipe.score(X_1d_test, y_1d_test)
x_plot = np.linspace(0, 10, 300).reshape(-1, 1)
y_plot = poly_pipe.predict(x_plot)
ax.scatter(X_1d.ravel(), y_1d, alpha=0.4, s=20, color='gray')
ax.plot(x_plot, y_plot, 'b-', linewidth=2)
ax.set_title(f'Degree={degree}\nTrain R²={train_r2:.3f}, Test R²={test_r2:.3f}')
plt.tight_layout()
plt.show()
Decision Tree and Random Forest Regression
python example
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
X, y = fetch_california_housing(return_X_y=True)
y = y * 100000
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Decision Tree
dt = DecisionTreeRegressor(max_depth=6, min_samples_leaf=10, random_state=42)
dt.fit(X_train, y_train)
evaluate_regression(y_test, dt.predict(X_test), 'Decision Tree (max_depth=6)')
# Random Forest (ensemble of decision trees)
rf = RandomForestRegressor(n_estimators=100, max_depth=10, min_samples_leaf=5,
n_jobs=-1, random_state=42)
rf.fit(X_train, y_train)
evaluate_regression(y_test, rf.predict(X_test), 'Random Forest')
# Gradient Boosting
gb = GradientBoostingRegressor(n_estimators=200, max_depth=4, learning_rate=0.05,
min_samples_leaf=5, random_state=42)
gb.fit(X_train, y_train)
evaluate_regression(y_test, gb.predict(X_test), 'Gradient Boosting')
# Feature Importance (Random Forest)
feature_importance = pd.Series(rf.feature_importances_,
index=fetch_california_housing().feature_names)
feature_importance = feature_importance.sort_values(ascending=False)
print(f"\nRandom Forest Feature Importance:\n{feature_importance}")
plt.figure(figsize=(10, 5))
feature_importance.plot(kind='bar', color='steelblue', alpha=0.8)
plt.title('Feature Importance - Random Forest')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()XGBoost Regression
bash example
pip install xgboostExample
import xgboost as xgb
from sklearn.model_selection import GridSearchCV
# XGBoost — often the best performing model for tabular data
xgb_model = xgb.XGBRegressor(
n_estimators=200,
max_depth=6,
learning_rate=0.05,
subsample=0.8,
colsample_bytree=0.8,
random_state=42,
n_jobs=-1
)
xgb_model.fit(X_train, y_train,
eval_set=[(X_test, y_test)],
verbose=50)
evaluate_regression(y_test, xgb_model.predict(X_test), 'XGBoost')
# Feature importance
xgb.plot_importance(xgb_model, max_num_features=8)
plt.title('XGBoost Feature Importance')
plt.show()
Model Comparison
python example
import pandas as pd
import numpy as np
models = {}
results = {}
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
X, y = fetch_california_housing(return_X_y=True)
y = y * 100000
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
models_to_compare = {
'Linear Regression': make_pipeline(StandardScaler(), LinearRegression()),
'Ridge (alpha=10)': make_pipeline(StandardScaler(), Ridge(alpha=10)),
'Lasso (alpha=100)': make_pipeline(StandardScaler(), Lasso(alpha=100, max_iter=10000)),
'Random Forest': RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1),
'Gradient Boosting': GradientBoostingRegressor(n_estimators=100, random_state=42),
}
comparison = []
for name, model in models_to_compare.items():
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
r2 = r2_score(y_test, y_pred)
rmse = np.sqrt(mean_squared_error(y_test, y_pred))
comparison.append({'Model': name, 'R²': round(r2, 4), 'RMSE': round(rmse, 0)})
df_compare = pd.DataFrame(comparison).sort_values('R²', ascending=False)
print("=== MODEL COMPARISON ===")
print(df_compare.to_string(index=False))Summary
In this lesson, you learned:
- ✅ Linear Regression and how to interpret coefficients
- ✅ Ridge (L2) regularization to reduce overfitting
- ✅ Lasso (L1) regularization for feature selection
- ✅ ElasticNet combining both L1 and L2
- ✅ Polynomial Regression for non-linear relationships
- ✅ Decision Tree and Random Forest Regression
- ✅ Gradient Boosting and XGBoost
- ✅ Model comparison and selection
- ✅ Regression evaluation metrics (RMSE, MAE, R², MAPE)
⚠️ Common Mistakes
1. Feature Scaling भूलना 🚫
Linear, Ridge, Lasso models को StandardScaler लगाना ज़रूरी है! बिना scaling के coefficients misleading होते हैं और convergence slow होती है।
2. Multicollinearity Ignore करना 📉
अगर features highly correlated हैं (जैसे AveRooms और AveBedrms), तो Linear Regression unstable coefficients देता है। Ridge/Lasso use करें या VIF check करें।
3. Train-Test Split से पहले Scaling करना ❌
python example
# ❌ WRONG — data leakage!
scaler.fit_transform(X) # full data पर fit
# ✅ CORRECT
scaler.fit(X_train) # सिर्फ train पर fit
scaler.transform(X_test) # test पर transform4. R² Score को Blindly Trust करना 🙈
High R² हमेशा अच्छा model नहीं — overfitting हो सकती है! Cross-validation और test set दोनों पर check करें।
5. Non-Linear Data पर Linear Regression Force करना 📏
अगर relationship non-linear है, तो Polynomial features add करें या Tree-based models use करें। Residual plots देखें — pattern नहीं होना चाहिए।
6. Hyperparameter Tuning Skip करना ⚙️
Default parameters rarely optimal होते हैं। GridSearchCV या RandomizedSearchCV use करें — especially alpha (Ridge/Lasso) और max_depth, n_estimators (Tree models) के लिए।
7. Outliers Handle ना करना 🎯
Linear Regression outliers से बहुत affected होता है। MAE vs RMSE compare करें — अगर RMSE >>> MAE, तो outliers impact कर रहे हैं। Robust scalers या tree-based models consider करें।
✅ Key Takeaways
- 🎯 Regression = Continuous prediction — जब target numerical हो (price, temperature, salary), regression use करो
- 📐 Linear Regression सबसे simple baseline है — हमेशा पहले इसे try करो
- 🛡️ Ridge (L2) coefficients shrink करता है, overfitting रोकता है — जब सभी features relevant हों
- ✂️ Lasso (L1) automatic feature selection करता है — coefficients exactly zero कर सकता है
- 🤝 ElasticNet = Lasso + Ridge — जब बहुत ज़्यादा features हों और कुछ correlated हों
- 📈 Polynomial Regression non-linear patterns capture करती है — लेकिन high degree = overfitting risk
- 🌲 Tree-based models (Random Forest, XGBoost) — scaling ज़रूरी नहीं, non-linear relationships handle करते हैं
- 🏆 XGBoost/Gradient Boosting usually best performance देते हैं tabular data पर
- 📊 Multiple metrics use करो — R², RMSE, MAE, MAPE — single metric से complete picture नहीं मिलती
- 🔄 Cross-Validation ज़रूरी है — single train-test split unreliable हो सकता है
❓ FAQ
Q1: Ridge और Lasso में कब क्या use करें?
Answer: अगर आपको लगता है कि सारे features important हैं, Ridge use करो (coefficients shrink होंगे but zero नहीं)। अगर कुछ features irrelevant हैं और automatic feature selection चाहिए, Lasso use करो। Unsure हो? ElasticNet try करो!
Q2: R² negative कब आता है?
Answer: जब model mean value predict करने से भी बुरा perform करे! इसका मतलब model completely wrong है — wrong features, bugs, या data leakage check करो। R² = 1 - (SS_res / SS_tot), अगर SS_res > SS_tot, तो negative हो जाता है।
Q3: RMSE vs MAE — कौन सा metric better है?
Answer: RMSE large errors को ज़्यादा penalize करता है (squared term), MAE सभी errors को equally treat करता है। अगर outliers handle नहीं कर सकते, MAE better है। अगर large errors ज़्यादा costly हैं (medical, finance), RMSE use करो।
Q4: Random Forest vs XGBoost — क्या difference है?
Answer: Random Forest parallel trees बनाता है (bagging), XGBoost sequential trees बनाता है (boosting — हर tree पिछले की mistakes correct करता है)। XGBoost usually better accuracy देता है लेकिन tuning ज़्यादा sensitive है। Random Forest simpler और robust है।
Q5: Feature Scaling सिर्फ linear models के लिए ज़रूरी है?
Answer: हाँ, primarily! Tree-based models (Decision Tree, RF, XGBoost) को scaling की ज़रूरत नहीं क्योंकि ये splits पर काम करते हैं, distances पर नहीं। Linear, Ridge, Lasso, ElasticNet, KNN, SVM — इन सबको scaling लगाओ।
Q6: Polynomial Regression में degree कैसे choose करें?
Answer: Cross-validation use करो! High degree = overfitting (train पर perfect, test पर terrible). Usually degree 2-3 sufficient होता है। Degree > 5 rarely useful — instead, tree-based models try करो।
Q7: Overfitting कैसे detect करें regression में?
Answer: Train R² vs Test R² compare करो। अगर train R² = 0.99 but test R² = 0.60, that's overfitting! Cross-validation scores भी check करो — high variance across folds = overfitting। Learning curves plot करो।
Q8: Production में कौन सा regression model recommend करोगे?
Answer: Depends on use case! Quick baseline → Linear Regression। Interpretability ज़रूरी → Ridge/Lasso। Best accuracy → XGBoost/LightGBM। Real-time prediction → simple models (inference speed matters)। Always start simple, then iterate!
🎯 Interview Questions
Q1: Linear Regression के assumptions क्या हैं?
Answer: Linear Regression 5 key assumptions पर based है:
- Linearity — X और y के बीच linear relationship हो
- Independence — observations independent हों (no autocorrelation)
- Homoscedasticity — residuals की variance constant हो across all X values
- Normality — residuals normally distributed हों
- No Multicollinearity — independent variables highly correlated न हों
Violation check: Residual plots, VIF (Variance Inflation Factor > 5 = problem), Durbin-Watson test (autocorrelation)।
Q2: Ridge और Lasso regression mathematically कैसे differ करते हैं?
Answer:
- Ridge: Loss = MSE + α × Σ(β²) — L2 penalty। Coefficients shrink towards zero but never exactly zero। Geometric interpretation: constraint region is a circle।
- Lasso: Loss = MSE + α × Σ|β| — L1 penalty। Coefficients can become exactly zero (sparse solution)। Geometric interpretation: constraint region is a diamond — corners पर solutions land करती हैं (zero coefficients)।
- Key difference: Lasso does feature selection, Ridge doesn't। Correlated features — Ridge better (Lasso randomly picks one)।
Q3: R², Adjusted R², RMSE, MAE में क्या difference है? कब कौन सा use करें?
Answer:
- R²: Explained variance proportion (0-1)। Problem: features add करने पर हमेशा बढ़ता है
- Adjusted R²: Features add करने पर penalize करता है — only improves if feature actually helps
- RMSE: Same unit as target, penalizes large errors more (squared)
- MAE: Same unit as target, robust to outliers (linear penalty)
- Use case: Reporting → R²/Adjusted R²। Model selection → RMSE/MAE। Outlier-sensitive → MAE prefer करो।
Q4: Bias-Variance Tradeoff explain करो regression context में।
Answer:
- High Bias (Underfitting): Model बहुत simple है (e.g., linear model on non-linear data)। Train और test दोनों पर poor performance।
- High Variance (Overfitting): Model बहुत complex है (e.g., degree-15 polynomial)। Train पर excellent, test पर terrible।
- Tradeoff: Bias decrease → Variance increase। Sweet spot find करना goal है।
- Solutions: Regularization (Ridge/Lasso), Cross-validation, Ensemble methods (RF reduces variance, Boosting reduces bias)।
Q5: Multicollinearity क्या है और कैसे handle करें?
Answer: जब independent variables एक-दूसरे से highly correlated हों (e.g., area_sqft और area_sqm)।
- Problems: Unstable coefficients, high standard errors, unreliable p-values
- Detection: Correlation matrix (>0.8), VIF (>5 concerning, >10 serious)
- Solutions: Remove one correlated feature, PCA use करो, Ridge regression (designed for this!), Domain knowledge से decide करो कौन सा feature रखना है।
Q6: Gradient Boosting internally कैसे काम करता है?
Answer: Step-by-step:
- Initial prediction = mean of target
- Calculate residuals (errors) = actual - predicted
- Fit a new tree on residuals (learn the mistakes)
- Update predictions: new_pred = old_pred + learning_rate × tree_prediction
- Repeat steps 2-4 for
n_estimatorsiterations
Key hyperparameters: learning_rate (small = slow but better), n_estimators (more trees = more complex), max_depth (usually 3-6)। Learning rate और n_estimators inversely related — small LR needs more trees।
Q7: Cross-Validation regression में कैसे implement करें और क्यों ज़रूरी है?
Answer:
python example
from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring='r2')
print(f"R² = {scores.mean():.4f} ± {scores.std():.4f}")Why: Single train-test split lucky/unlucky हो सकता है। CV gives reliable estimate — mean score = expected performance, std = stability। K=5 or K=10 standard है। Time-series data: use TimeSeriesSplit (never shuffle!)।
Q8: Feature Importance कैसे interpret करें tree-based models में?
Answer:
- Impurity-based (default in sklearn): Each split पर impurity (MSE) कितना decrease हुआ, averaged across all trees। Fast but biased towards high-cardinality features।
- Permutation importance: Feature values shuffle करो → performance drop = importance। Unbiased but slower।
- SHAP values: Game theory based — each feature का individual prediction पर contribution। Best for explaining individual predictions।
Caution: Correlated features share importance — individual importance misleading हो सकती है।
Q9: Regularization का alpha/lambda कैसे tune करें?
Answer:
alphaबड़ा → ज़्यादा regularization → simpler model (high bias, low variance)alphaछोटा → कम regularization → complex model (low bias, high variance)- Best practice:
RidgeCV,LassoCV, याGridSearchCVuse करो - Search range: Log scale पर [0.001, 0.01, 0.1, 1, 10, 100, 1000]
- ElasticNet:
l1_ratioभी tune करो (0=Ridge, 1=Lasso)
Q10: Real-world regression project का workflow क्या होना चाहिए?
Answer: Complete pipeline:
- EDA: Target distribution, correlations, missing values check करो
- Preprocessing: Handle missing values, encode categoricals, scale numericals
- Baseline: Simple LinearRegression → benchmark set करो
- Feature Engineering: Domain knowledge, interactions, polynomial features
- Model Selection: Multiple algorithms try करो (Linear → Tree → Boosting)
- Hyperparameter Tuning: GridSearchCV/RandomizedSearchCV with CV
- Evaluation: Multiple metrics, residual analysis, error distribution
- Production: Pipeline बनाओ (preprocessing + model), save with joblib, API deploy
*Next Lesson: Classification →*
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Regression Algorithms.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, machine, learning, regression
Related Python Master Course Topics
Continue learning this concept
Machine LearningClassification AlgorithmsLearn classification algorithms in Python — Logistic Regression, Decision Trees, Random Forest, SVM, KNN, Naive Bayes, and Gradient Boosting, with metrics, confusion matrices, ROC curves, and class imbalance handling.Machine LearningClustering AlgorithmsLearn unsupervised clustering algorithms in Python — K-Means, Hierarchical Clustering, DBSCAN, and Gaussian Mixture Models, with evaluation metrics, the elbow method, and customer segmentation project.Machine LearningIntroduction to Machine LearningLearn the fundamentals of machine learning — what it is, types of ML (supervised, unsupervised, reinforcement), key concepts, the ML workflow, and how to set up your Python ML environment.Machine LearningMachine Learning Project: Churn PredictionBuild a complete end-to-end machine learning project — predict customer churn using real-world techniques: EDA, feature engineering, model training, evaluation, hyperparameter tuning, and deployment-ready pipeline.Machine LearningModel Evaluation and SelectionMaster ML model evaluation — cross-validation strategies, evaluation metrics for regression and classification, overfitting detection, hyperparameter tuning, model selection best practices, and model persistence with joblib.Machine LearningSave pipeline (includes preprocessing + model)Learn about save pipeline (includes preprocessing + model) in Python Programming with detailed explanations and examples.