AI Notes

AI Interview Cheatsheet

Quick reference for AI interviews. Algorithms, formulas. Interview 2024.

Introduction

Technical interviews for AI and machine learning roles test both theoretical understanding and practical problem-solving ability. This cheatsheet organizes the most frequently asked concepts, formulas, and algorithms into a format designed for rapid review before interviews. Each section covers what interviewers typically expect you to know, with concise explanations and key differentiators between similar concepts.

Machine Learning Fundamentals

Bias-Variance Tradeoff

- Fix	More features, complex model, less regularization
- Fix	More data, regularization, simpler model, dropout
Sweet spot	Model complex enough to capture patterns

Regularization Quick Reference

L1 (Lasso)	Loss + λ Σ\|w_i\|
Effect	Drives weights to exactly zero (feature selection)
Use when	You suspect many irrelevant features
L2 (Ridge)	Loss + λ Σ w_i²
Effect	Shrinks all weights toward zero (none exactly zero)
Use when	All features potentially relevant
Elastic Net	Loss + λ₁Σ\|w_i\| + λ₂Σw_i²
Combines both	Some feature selection + weight shrinkage
Dropout (neural networks)	Randomly zero p% of neurons during training
Effect	Ensemble of sub-networks, prevents co-adaptation
Typical p	0.2-0.5 for hidden layers

Classification vs Regression

Classification	Predict discrete class labels
Algorithms	Logistic Regression, SVM, Decision Trees, Neural Nets
Metrics	Accuracy, Precision, Recall, F1, AUC-ROC
Loss	Cross-entropy
Regression	Predict continuous values
Algorithms	Linear Regression, SVR, Decision Trees, Neural Nets
Metrics	MSE, RMSE, MAE, R²
Loss	Mean Squared Error

Deep Learning Quick Reference

Architecture Selection Guide

Data type	Architecture:
Tabular data	Gradient Boosting (XGBoost) or MLP
Images	CNN (ResNet, EfficientNet)
Text/Sequences	Transformer (BERT for understanding, GPT for generation)
Time series	LSTM/GRU or Temporal CNN or Transformer
Graphs	GNN (Graph Neural Networks)
Audio	CNN on spectrograms or wav2vec

Transformer Key Concepts

Self-Attention	Each token attends to all other tokens
Multi-Head	Multiple attention heads capture different relationships
BERT	Bidirectional encoder, masked language model
Use for	Classification, NER, QA, sentence similarity
GPT	Autoregressive decoder, left-to-right generation
Use for	Text generation, summarization, translation

Common Interview Questions and Answers

Q	"Why does batch normalization help?"
A	Reduces internal covariate shift (layer input distributions change
Q	"Explain vanishing gradient problem"
A	In deep networks with sigmoid/tanh, gradients shrink exponentially
learn. Solutions	ReLU, residual connections, batch norm, LSTM gates.
Q	"What's the difference between GAN and VAE?"
A	GAN: Two networks compete (generator vs discriminator). Sharper
VAE	Encoder-decoder with latent distribution. Stable training,
Q	"How does dropout work at test time?"
A	At test time, ALL neurons are active (no dropout). Weights are

System Design for ML

1. Clarify requirements	What metric to optimize? Latency constraints?
2. Data	Sources, volume, labeling, preprocessing pipeline
3. Feature engineering	What signals are available?
4. Model selection	Start simple (baseline), then add complexity
5. Training	Infrastructure, hyperparameter tuning, validation strategy
6. Serving	Batch vs real-time, model size, inference optimization
7. Monitoring	Data drift, model degradation, A/B testing
Example	"Design a spam classifier"
Metric	Precision (don't mark good email as spam)
Features	Text content, sender reputation, link analysis, metadata
Baseline	Logistic regression on TF-IDF (fast to deploy)
Advanced	Fine-tuned BERT (better accuracy, higher latency)
Serving: Real-time (email arrives	classify immediately)
Monitor	Track false positive rate daily, retrain monthly

Optimization and Training

Optimizer comparison

SGD + Momentum: Good generalization, needs LR scheduling

Adam: Fast convergence, good default, may generalize worse

AdamW: Adam with proper weight decay (recommended for transformers)

Learning rate strategies

Warm-up: Start low, increase linearly for first N steps

Cosine decay: Gradually decrease following cosine curve

Step decay: Reduce by factor every N epochs

One-cycle: Increase then decrease (super-convergence)

Hyperparameter priority (tune in this order)

1. Learning rate (most impactful)

2. Batch size (affects generalization and training dynamics)

3. Architecture depth/width

4. Regularization strength

5. Other (momentum, warmup steps, etc.)

Quick Recall: Key Numbers

ImageNet accuracy milestones

Human: ~5% top-5 error

AlexNet (2012): 15.3%

ResNet (2015): 3.57%

Typical model sizes

BERT-base: 110M parameters

GPT-3: 175B parameters

GPT-4: ~1.8T parameters (estimated)

Training data sizes

BERT: 16GB text (Wikipedia + BookCorpus)

GPT-3: 570GB text (Common Crawl + books)

Stable Diffusion: 5B image-text pairs

Rule of thumb

Parameters ≈ 10× training examples for good generalization

More data almost always helps (until diminishing returns)

Summary

Success in AI interviews requires connecting theoretical knowledge with practical intuition. Know the formulas but also understand when and why each technique is appropriate. Practice explaining concepts simply: if you can teach it to a non-expert in 30 seconds, you truly understand it. Focus on tradeoffs and design decisions rather than memorizing exact implementations.

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for AI Interview Cheatsheet.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, cheatsheet, ai interview cheatsheet

Continue learning this concept

Interview PreparationAI Interview QuestionsMaster AI interview prep. Algorithms, deep learning. Interview 2024.Interview PreparationDeep Learning Interview Questions — Artificial IntelligenceMaster DL interview prep. Networks, training. Interview 2024.Interview PreparationLLM & RAG Interview QuestionsLearn LLM interview prep. Language models, RAG. Interview 2024.Interview PreparationML Interview QuestionsComplete ML interview prep. Algorithms, optimization. Interview 2024.Ai Agents And LlmsAgentic AI ProjectsBuild practical agentic AI systems with multi-tool orchestration, autonomous planning, and real-world applications.Ai Agents And LlmsAI Agents: Architecture and ImplementationComprehensive guide to agent design, architecture, autonomous decision-making, and implementation patterns.

PreviousVoice Assistant Project NextAI Interview Questions