ML Notes
HubAI vs ML vs Deep Learning — Machine LearningApplications of Machine LearningHistory of Machine LearningMachine Learning RoadmapWhat is Machine Learning?Calculus for Machine LearningData Preprocessing BasicsLinear Algebra for Machine LearningProbability Basics for Machine LearningPython for Machine LearningStatistics Basics for Machine LearningBias vs Variance TradeoffModel Evaluation OverviewOverfitting and UnderfittingReinforcement Learning Fundamentals — Machine LearningSupervised LearningTraining vs Testing DataTypes of Machine LearningUnsupervised LearningData Cleaning — Machine LearningData Collection for Machine LearningData Visualization for MLEncoding Categorical DataFeature EngineeringFeature ScalingHandling Missing ValuesTrain-Test SplitElastic Net RegressionLasso Regression (L1 Regularization)Multiple Linear RegressionPolynomial RegressionIntroduction to RegressionRegression ProjectsRidge Regression (L2 Regularization)Simple Linear RegressionIntroduction to ClassificationClassification ProjectsDecision TreesK-Nearest Neighbors (KNN)Logistic RegressionNaive Bayes ClassifierRandom Forest ClassificationSupport Vector Machine (SVM)XGBoost for ClassificationClustering in Machine Learning: Complete Guide to Unsupervised LearningClustering Projects: Real-World Applications and Implementation GuidesDBSCAN ClusteringGaussian Mixture Models (GMM): Probabilistic Clustering for Advanced Data AnalysisHierarchical ClusteringK-Means ClusteringDimensionality Reduction: Comprehensive Guide to Feature Reduction TechniquesFeature Selection Techniques: Choosing Informative Variables for ML ModelsLinear Discriminant Analysis (LDA): Supervised Dimensionality ReductionPrincipal Component Analysis (PCA)t-SNE: Non-Linear Dimensionality Reduction for Data VisualizationUMAP: Uniform Manifold Approximation and Projection for Modern Data VisualizationAdaBoost: Adaptive Boosting Algorithm Explained with ExamplesBagging: Bootstrap Aggregating for Reducing Model VarianceBoosting: Sequential Ensemble Learning for Bias and Variance ReductionEnsemble Learning: Complete Guide to Combining ML Models for Better PredictionsGradient Boosting: Building Powerful ML Models Through Sequential OptimizationRandom Forest: Complete Guide to Tree Ensembles and Feature ImportanceStacking: Advanced Ensemble Method Using Meta-Learners for Superior ML PredictionsActivation Functions in Neural NetworksBackpropagation: Complete Algorithm for Training Neural NetworksIntroduction to Deep Learning: Neural Networks, Architectures, and ApplicationsDeep Learning Projects: Hands-On Applications and Case StudiesNeural Networks Basics: Perceptron, Architecture, and TrainingPerceptron: Foundational Single-Neuron Classifier for Binary ClassificationPyTorch Basics: Building and Training Neural NetworksTensorFlow Basics: Building and Training Neural NetworksIntroduction to Natural Language Processing: Fundamentals and ApplicationsNLP Projects: Practical Applications from Sentiment Analysis to Machine TranslationStemming and Lemmatization: Reducing Words to Root Forms in NLPText Preprocessing: Essential Steps for NLP ApplicationsTF-IDF: Term Frequency-Inverse Document Frequency for Text Feature ExtractionTokenization in NLP: Breaking Text into Meaningful UnitsTransformers: The Architecture Behind Modern NLP ModelsWord Embeddings: Representing Words as Vectors for NLPIntroduction to Computer VisionComputer Vision ProjectsImage Classification: Building and Deploying CNN ModelsImage Processing BasicsImage Segmentation — Machine LearningObject DetectionOpenCV BasicsAgents and Environments in RLDeep Q-Networks (DQN)Policy Gradient Methods — Machine LearningQ-Learning AlgorithmReinforcement Learning: IntroductionReinforcement Learning ProjectsAccuracy, Precision, Recall: Classification Metrics Explained for ML InterviewConfusion Matrix: Complete Guide to Classification EvaluationCross-Validation: Comprehensive Guide to Robust Model EvaluationF1 Score: Balanced Classification Metric for Imbalanced DataGrid Search: Complete Guide to Hyperparameter Tuning in Machine LearningHyperparameter Tuning: Complete Guide to Optimizing Machine Learning ModelsModel Selection: Choosing the Best Algorithm for Your ML ProblemROC and AUC: Complete Guide to Classification Threshold EvaluationBuild an AI Assistant with LLMsBuild a Conversational ChatbotCustomer Churn PredictionFace Recognition System — Machine LearningHouse Price PredictionRecommendation System ProjectSentiment Analysis Project — Machine LearningSpam Email ClassifierML Case Study Questions for InterviewsML Coding Questions for Technical InterviewsMachine Learning Interview CheatsheetEssential Machine Learning Interview QuestionsStatistics for Machine Learning InterviewsDatasets for ML PracticeImportant ML FormulasMachine Learning GlossaryRecommended ML Books and CoursesCI/CD for Machine LearningDocker for Machine LearningFastAPI for Machine Learning APIsFlask API for Machine LearningMLOps: IntroductionModel Deployment StrategiesML Model Monitoring & Observability
Different approaches to deploying ML models: batch, real-time, shadow, canary, blue-green, and A/B testing.
Different deployment approaches serve different needs. Choose based on latency requirements, risk tolerance, and traffic patterns.
Deployment Types
1. Batch Deployment
Process data in batches, generate predictions offline, store results.
Example
Schedule (hourly/daily)
↓
Load batch of data
↓
Generate predictions
↓
Store results in DB
↓
Serve from DB to users
Use when:
- Latency not critical (can wait hours)
- Process large datasets efficiently
- Lower computational cost
- Predictions don't need to be real-time
Example: Daily email recommendations
python example
def batch_predict_emails():
all_users = load_all_users()
for batch in chunks(all_users, batch_size=1000):
user_features = prepare_features(batch)
predictions = model.predict(user_features)
store_predictions(batch, predictions)
# Send emails tomorrow morning2. Real-time/Online Deployment
Generate predictions immediately on request, low latency required.
Example
Request
↓
Load features (may call API)
↓
Model prediction
↓
Return in <100ms
Use when:
- Latency critical (must respond quickly)
- Unpredictable request patterns
- Individual predictions needed
- Can't batch process
Example: Credit card fraud detection
python example
@app.post("/check_transaction")
def check_transaction(transaction):
risk_score = model.predict(transaction)
if risk_score > 0.8:
return {"status": "blocked", "reason": "fraud_risk"}
else:
return {"status": "approved"}Latency considerations:
- <10ms: Very fast (cache possible)
- 10-100ms: Fast (acceptable for most)
- 100-1000ms: Acceptable for some user interactions
- >1000ms: Too slow for interactive use
3. Shadow Deployment
Run new model alongside current, capture predictions but don't use them.
Example
User Request
↓
/--------\
| |
Current Model (USED) New Model (LOGGED ONLY)
| |
\--------/
Process:
- Deploy new model to prod environment
- Run both models on same data
- Compare predictions, log differences
- Monitor performance without risking users
Benefits:
- Test new model in real production environment
- Compare predictions with actual data
- No risk to end users
- Can run indefinitely until confident
Implementation:
python example
class ShadowDeployment:
def __init__(self, current_model, new_model, logger):
self.current_model = current_model
self.new_model = new_model
self.logger = logger
def predict(self, X):
# Production prediction (current)
current_pred = self.current_model.predict(X)
# Shadow prediction (new, logged)
new_pred = self.new_model.predict(X)
# Log comparison
difference = np.abs(current_pred - new_pred).mean()
self.logger.info(f"Shadow diff: {difference:.4f}")
# Return current (don't use new yet)
return current_pred4. Canary Deployment
Gradually increase traffic to new model, monitor closely, roll back if issues.
| ├─ Hour 1-2: 5% traffic | New Model, 95% → Current |
| ├─ Hour 3-4: 10% traffic | New Model |
| ├─ Hour 5-24 | 25%, 50%, 75% gradually |
| └─ If healthy: 100% traffic | New Model |
Steps:
- Deploy new model to production
- Route small percentage (5%) of traffic to new model
- Monitor metrics carefully
- Gradually increase traffic: 10% → 25% → 50% → 100%
- Keep old model ready for instant rollback
Benefits:
- Low-risk rollout
- Early detection of issues
- Smooth transition
- Easy rollback if problems
Implementation:
python example
def canary_deployment(user_id, X):
"""Route percentage of traffic to new model"""
canary_percentage = get_current_canary_traffic() # 5%, 10%, 25%...
# Consistent assignment (same user always routed same way)
hash_value = int(hashlib.md5(str(user_id).encode()).hexdigest(), 16)
user_bucket = hash_value % 100
if user_bucket < canary_percentage:
prediction, model_used = new_model.predict(X), 'new'
else:
prediction, model_used = current_model.predict(X), 'current'
# Log which model was used
log_deployment(user_id, model_used, prediction)
return prediction5. Blue-Green Deployment
Two identical production environments, switch between them.
BLUE (Current)
├─ Model v1
├─ DB connections
└─ API endpoints
GREEN (New)
├─ Model v2
├─ DB connections (same)
└─ API endpoints
Router: Initially points to BLUE
After verification: Switch to GREEN
Process:
- Deploy new model to green environment
- Run full test suite on green
- Switch router to direct traffic to green
- Keep blue as instant rollback option
- After stable, retire blue
Benefits:
- Atomic switch (no gradual transition)
- Easy rollback
- Full environment tested
6. A/B Testing Deployment
Compare two models on random samples, measure statistical significance.
| ├─ 50% Control Group | Model A (current) |
| └─ 50% Treatment Group | Model B (new) |
| Measure | Click rate, conversion, engagement, revenue |
| Duration | 2 weeks (enough for statistical significance) |
Process:
- Randomly assign users to control/treatment
- Serve different models
- Collect metrics for both groups
- Statistical test (t-test, chi-square)
- If significant improvement: deploy to all
Implementation:
Example
def ab_test_deployment(user_id, X):
"""A/B test: assign to control or treatment"""
# Deterministic: same user always in same group
hash_value = int(hashlib.md5(str(user_id).encode()).hexdigest(), 16)
if hash_value % 100 < 50:
prediction = model_a.predict(X)
group = 'control'
else:
prediction = model_b.predict(X)
group = 'treatment'
# Log for analysis
log_ab_test(user_id, group, prediction)
return prediction
# Analysis after 2 weeks
control_metrics = analyze_group('control')
treatment_metrics = analyze_group('treatment')
improvement = (treatment_metrics['ctr'] - control_metrics['ctr']) / control_metrics['ctr']
print(f"CTR improvement: {improvement:.1%}")
# Statistical test
t_stat, p_value = ttest(control_metrics['data'], treatment_metrics['data'])
if p_value < 0.05:
print("Significant improvement - deploy to all")
Recommended Deployment Flow
| ├─ If healthy | increase to 10% |
| ├─ 10% | 25% → 50% → 100% |
| ├─ Each stage | 2-4 hours |
Risk Management
Metrics to Monitor
| Metric | Target | Alert Threshold |
|---|---|---|
| Error rate | <0.1% | >0.5% |
| Latency p99 | <100ms | >500ms |
| Model accuracy | baseline | -5% drop |
| Data quality | baseline | Any drift |
Rollback Triggers
Automatic rollback if:
- Error rate spikes >1%
- Latency p99 > 5 seconds
- Model accuracy drops >10%
- Data quality check fails
- Manual override by operator
Example
def check_rollback_conditions():
metrics = get_current_metrics()
if metrics['error_rate'] > 0.01:
trigger_rollback("Error rate spike")
if metrics['latency_p99'] > 5.0:
trigger_rollback("Latency spike")
if metrics['accuracy'] < baseline * 0.9:
trigger_rollback("Accuracy drop")
if detect_data_drift():
trigger_rollback("Data drift detected")
Summary
Deployment strategies:
- Batch: When latency not critical
- Real-time: When fast response needed
- Shadow: Test in production safely
- Canary: Gradual rollout with monitoring
- Blue-Green: Atomic switching
- A/B Test: Statistical comparison
Choose based on risk tolerance, latency needs, and business requirements.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Model Deployment Strategies.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Machine Learning topic.
Search Terms
machine-learning, machine learning, machine, learning, mlops, model, deployment, model deployment strategies
Related Machine Learning Topics
Continue learning this concept
MlopsML Model Monitoring & ObservabilityMonitoring ML models in production: performance metrics, data drift detection, alerting, and logging.MlopsCI/CD for Machine LearningImplementing continuous integration and continuous deployment pipelines for ML models including testing, validation, and automated deployment.MlopsDocker for Machine LearningContainerizing ML models and workflows using Docker for reproducibility, scalability, and deployment.MlopsFastAPI for Machine Learning APIsBuilding high-performance ML APIs with FastAPI, async support, and automatic documentation.MlopsFlask API for Machine LearningBuilding ML model serving APIs with Flask, request handling, and deployment.MlopsMLOps: IntroductionIntroduction to MLOps: productionizing ML systems, monitoring, deployment, and lifecycle management.