AI Notes

AI Interview Questions

Master AI interview prep. Algorithms, deep learning. Interview 2024.

Fundamental Concepts

Q1: What is the difference between AI, Machine Learning, and Deep Learning?

Answer: AI is the broadest concept—any system that exhibits intelligent behavior. Machine Learning is a subset of AI where systems learn patterns from data without explicit programming. Deep Learning is a subset of ML using neural networks with many layers (deep architectures). Think of concentric circles: AI ⊃ ML ⊃ DL.

AI	Rule-based expert systems, search algorithms, planning
ML	Decision trees, SVM, random forests, neural networks
DL	CNNs, RNNs, Transformers, GANs (many-layer neural nets)

Q2: Explain the bias-variance tradeoff.

Answer: Models can err in two ways. Bias is systematic error from overly simple assumptions—the model consistently misses the pattern (underfitting). Variance is sensitivity to training data fluctuations—the model memorizes noise (overfitting).

High bias + Low variance	Linear regression on nonlinear data
Low bias + High variance	Deep net on small dataset
Goal	Sweet spot where total error (bias² + variance) is minimized
Reduce bias	More complex model, more features
Reduce variance	Regularization, more data, dropout, ensembles

Q3: What are the types of machine learning?

Supervised Learning: Labeled data	learn input→output mapping
Classification	spam/not-spam, cat/dog
Regression	house price, temperature prediction
Unsupervised Learning: Unlabeled data	find structure
Clustering	customer segmentation (K-means)
Dimensionality reduction	PCA, t-SNE
Anomaly detection	fraud detection
Reinforcement Learning	Learn by trial and error
Examples	game playing, robot control
Semi-supervised	Mix of labeled + unlabeled data
Self-supervised	Generate labels from data itself (BERT, GPT)

Q4: Explain gradient descent and its variants.

Core idea	Minimize loss by iteratively moving against the gradient
Batch GD: Compute gradient over ALL data	one update
Stochastic GD: Gradient from ONE sample	update immediately
Mini-batch GD	Gradient from batch of 32-256 samples
Best of both	stable enough + fast enough
Momentum	Accumulate velocity (β=0.9)
Adam	Adaptive learning rates + momentum (default choice)
AdamW	Adam with proper weight decay (for transformers)

Q5: What is regularization and why is it needed?

Problem	Complex models overfit (memorize training data)
Solution	Regularization adds constraints to prevent overfitting
L1 (Lasso)	Loss + λΣ\|wᵢ\|
Drives some weights to exactly zero	feature selection
L2 (Ridge)	Loss + λΣwᵢ²
Shrinks all weights toward zero	smooth decision boundaries
Dropout	Randomly zero neurons during training (p=0.5)
Forces redundant representations	ensemble-like effect
Early stopping	Stop training when validation loss increases
Data augmentation	Artificially expand training set

Q6: Explain the attention mechanism.

Core intuition	Not all inputs are equally relevant
Query	What am I looking for?
Key	What do I contain?
Value	What information do I provide?
Multi-head	Multiple parallel attention functions
Concat and project	rich contextualized features

Q7: Compare CNNs and Transformers for vision.

CNNs

- Local receptive fields (inductive bias for spatial locality)

- Translation equivariant (same features anywhere in image)

- Hierarchical features (edges → parts → objects)

- Work well with limited data

- Efficient (sparse connections)

Transformers (ViT)

- Global receptive field from first layer

- No spatial inductive bias (learn from data)

- Need more data to match CNNs (21k+ ImageNet)

- But surpass CNNs at scale

- Self-attention captures long-range dependencies

Hybrid approach: CNN early layers + Transformer later

Swin Transformer: windowed attention with hierarchical merging

Q8: What is the vanishing gradient problem?

After 10 layers	0.25^10 ≈ 0.000001 (vanished!)
1. ReLU activation	derivative = 1 for positive inputs
2. Residual connections	gradient shortcut paths (ResNet)
3. Batch normalization	keeps activations in good range
4. LSTM gates	gradient highway through cell state
5. Careful initialization	Xavier/He initialization

Q9: Explain transfer learning.

Concept	Knowledge learned on one task helps another task
Why it works	Low-level features are universal
Image models	edges, textures (reusable everywhere)
Language models	grammar, semantics (reusable for any text task)

Q10: What are GANs and how do they work?

Generator (G)	Creates fake data from random noise
Discriminator (D)	Classifies data as real or fake
G tries to fool D	generates more realistic data
D tries to catch G	gets better at spotting fakes
At equilibrium	G produces data indistinguishable from real
Applications	Image generation, style transfer, super-resolution
Challenges	Mode collapse, training instability

System Design Questions

Q11: Design a recommendation system.

Architecture

Content-based: Match user preferences to item features

Collaborative filtering: Users who liked X also liked Y

Hybrid: Combine both (Netflix Prize winner)

Deep learning approach

User embedding + Item embedding → dot product → relevance score

Two-tower architecture: separate user/item encoders

Training: (user, item, clicked/not) pairs

Serving: Approximate nearest neighbors for real-time retrieval

Challenges: Cold start (new users/items), scalability, diversity

Behavioral Questions

How do you approach a new ML problem?

Example

1. Understand the business problem (not just the ML problem) 2. Define success metrics (accuracy? latency? fairness?) 3. Explore data (distributions, missing values, quality) 4. Establish a simple baseline (logistic regression, rules) 5. Iterate: feature engineering → model selection → tuning 6. Evaluate properly (held-out test set, cross-validation) 7. Deploy, monitor, iterate (ML is never "done")

Interview Tips

Example

- Always clarify the question before answering - Start with the big picture, then dive into details - Use concrete examples and numbers when possible - Acknowledge trade-offs (no solution is perfect) - Mention practical considerations (compute, data, latency) - Be honest about what you don't know - Show enthusiasm for learning and problem-solving

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for AI Interview Questions.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, questions, ai interview questions

Continue learning this concept

Interview PreparationAI Interview CheatsheetQuick reference for AI interviews. Algorithms, formulas. Interview 2024.Interview PreparationDeep Learning Interview Questions — Artificial IntelligenceMaster DL interview prep. Networks, training. Interview 2024.Interview PreparationLLM & RAG Interview QuestionsLearn LLM interview prep. Language models, RAG. Interview 2024.Interview PreparationML Interview QuestionsComplete ML interview prep. Algorithms, optimization. Interview 2024.Ai Agents And LlmsAgentic AI ProjectsBuild practical agentic AI systems with multi-tool orchestration, autonomous planning, and real-world applications.Ai Agents And LlmsAI Agents: Architecture and ImplementationComprehensive guide to agent design, architecture, autonomous decision-making, and implementation patterns.

PreviousAI Interview Cheatsheet NextDeep Learning Interview Questions — Artificial Intelligence

AI Notes

AI Interview Questions

Master AI interview prep. Algorithms, deep learning. Interview 2024.

Fundamental Concepts

Q1: What is the difference between AI, Machine Learning, and Deep Learning?

AI	Rule-based expert systems, search algorithms, planning
ML	Decision trees, SVM, random forests, neural networks
DL	CNNs, RNNs, Transformers, GANs (many-layer neural nets)

Q2: Explain the bias-variance tradeoff.

High bias + Low variance	Linear regression on nonlinear data
Low bias + High variance	Deep net on small dataset
Goal	Sweet spot where total error (bias² + variance) is minimized
Reduce bias	More complex model, more features
Reduce variance	Regularization, more data, dropout, ensembles

Q3: What are the types of machine learning?

Supervised Learning: Labeled data	learn input→output mapping
Classification	spam/not-spam, cat/dog
Regression	house price, temperature prediction
Unsupervised Learning: Unlabeled data	find structure
Clustering	customer segmentation (K-means)
Dimensionality reduction	PCA, t-SNE
Anomaly detection	fraud detection
Reinforcement Learning	Learn by trial and error
Examples	game playing, robot control
Semi-supervised	Mix of labeled + unlabeled data
Self-supervised	Generate labels from data itself (BERT, GPT)

Q4: Explain gradient descent and its variants.

Core idea	Minimize loss by iteratively moving against the gradient
Batch GD: Compute gradient over ALL data	one update
Stochastic GD: Gradient from ONE sample	update immediately
Mini-batch GD	Gradient from batch of 32-256 samples
Best of both	stable enough + fast enough
Momentum	Accumulate velocity (β=0.9)
Adam	Adaptive learning rates + momentum (default choice)
AdamW	Adam with proper weight decay (for transformers)

Q5: What is regularization and why is it needed?

Problem	Complex models overfit (memorize training data)
Solution	Regularization adds constraints to prevent overfitting
L1 (Lasso)	Loss + λΣ\|wᵢ\|
Drives some weights to exactly zero	feature selection
L2 (Ridge)	Loss + λΣwᵢ²
Shrinks all weights toward zero	smooth decision boundaries
Dropout	Randomly zero neurons during training (p=0.5)
Forces redundant representations	ensemble-like effect
Early stopping	Stop training when validation loss increases
Data augmentation	Artificially expand training set

Q6: Explain the attention mechanism.

Core intuition	Not all inputs are equally relevant
Query	What am I looking for?
Key	What do I contain?
Value	What information do I provide?
Multi-head	Multiple parallel attention functions
Concat and project	rich contextualized features

Q7: Compare CNNs and Transformers for vision.

CNNs

- Local receptive fields (inductive bias for spatial locality)

- Translation equivariant (same features anywhere in image)

- Hierarchical features (edges → parts → objects)

- Work well with limited data

- Efficient (sparse connections)

Transformers (ViT)

- Global receptive field from first layer

- No spatial inductive bias (learn from data)

- Need more data to match CNNs (21k+ ImageNet)

- But surpass CNNs at scale

- Self-attention captures long-range dependencies

Hybrid approach: CNN early layers + Transformer later

Swin Transformer: windowed attention with hierarchical merging

Q8: What is the vanishing gradient problem?

After 10 layers	0.25^10 ≈ 0.000001 (vanished!)
1. ReLU activation	derivative = 1 for positive inputs
2. Residual connections	gradient shortcut paths (ResNet)
3. Batch normalization	keeps activations in good range
4. LSTM gates	gradient highway through cell state
5. Careful initialization	Xavier/He initialization

Q9: Explain transfer learning.

Concept	Knowledge learned on one task helps another task
Why it works	Low-level features are universal
Image models	edges, textures (reusable everywhere)
Language models	grammar, semantics (reusable for any text task)

Q10: What are GANs and how do they work?

Generator (G)	Creates fake data from random noise
Discriminator (D)	Classifies data as real or fake
G tries to fool D	generates more realistic data
D tries to catch G	gets better at spotting fakes
At equilibrium	G produces data indistinguishable from real
Applications	Image generation, style transfer, super-resolution
Challenges	Mode collapse, training instability

System Design Questions

Q11: Design a recommendation system.

Architecture

Content-based: Match user preferences to item features

Collaborative filtering: Users who liked X also liked Y

Hybrid: Combine both (Netflix Prize winner)

Deep learning approach

User embedding + Item embedding → dot product → relevance score

Two-tower architecture: separate user/item encoders

Training: (user, item, clicked/not) pairs

Serving: Approximate nearest neighbors for real-time retrieval

Challenges: Cold start (new users/items), scalability, diversity

Behavioral Questions

How do you approach a new ML problem?

Example

Interview Tips

Example

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for AI Interview Questions.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, questions, ai interview questions

Continue learning this concept

PreviousAI Interview Cheatsheet NextDeep Learning Interview Questions — Artificial Intelligence