AI Notes

Deep Learning Interview Questions — Artificial Intelligence

Master DL interview prep. Networks, training. Interview 2024.

Architecture Questions

Q1: Explain the Transformer architecture in detail.

Components

Encoder: Self-attention + FFN, bidirectional

Decoder: Masked self-attention + cross-attention + FFN

Self-Attention

Q, K, V = input × W_Q, W_K, W_V

Attention = softmax(QK^T/√d_k) × V

Multi-head: h parallel attention functions (h=8 typically)

Each head: d_k = d_model/h (64 for d_model=512)

Concat outputs → linear projection

FFN: Two-layer MLP applied per position

FFN(x) = ReLU(xW₁ + b₁)W₂ + b₂

Inner dimension 4× model dimension (2048 for d_model=512)

Residual connections + LayerNorm after each sublayer

Positional encoding: sinusoidal or learned

Q2: What are the key differences between BERT, GPT, and T5?

BERT (Encoder-only)

Bidirectional attention (sees full context)

Pre-training: MLM + NSP

Best for: Classification, NER, QA (understanding)

Cannot generate text naturally

GPT (Decoder-only)

Causal (left-to-right) attention

Pre-training: Next token prediction

Best for: Text generation, dialogue, code

Can do classification via prompting

T5 (Encoder-Decoder)

Full transformer (encoder + decoder)

Text-to-text framework (everything is seq2seq)

Best for: Translation, summarization, multi-task

"Translate English to French: ..." → "..."

Q3: How does batch normalization work and why is it useful?

For each mini-batch

μ_B = mean(batch)

σ²_B = variance(batch)

x̂ = (x - μ_B) / √(σ²_B + ε)

output = γx̂ + β (learned scale and shift)

Benefits

- Stabilizes training (prevents internal covariate shift)

- Allows higher learning rates (faster training)

- Acts as regularizer (batch statistics add noise)

vs. Layer Normalization:

BN: normalize across batch (good for CNNs)

LN: normalize across features (good for Transformers, RNNs)

LN doesn't depend on batch size → works for any batch/sequence

Q4: Explain residual connections and why they help.

Residual	output = F(x) + x (skip connection)
Without residual	network must learn identity if layer is useless
With residual	if F(x)=0 is best, output = 0 + x = x automatically
Used in	ResNets, Transformers, any deep architecture

Q5: What is the difference between model parallelism and data parallelism?

Data Parallelism

Same model copied to N GPUs

Each GPU processes different batch of data

Gradients averaged across GPUs, then applied

Scales to: large batches

Limitation: model must fit on one GPU

Model Parallelism

Model split across N GPUs

Each GPU holds different layers/parts

Data passes through GPUs sequentially

Scales to: very large models (GPT-3: 175B params)

Limitation: sequential dependency, GPU utilization

Pipeline Parallelism

Model split into stages across GPUs

Multiple mini-batches in pipeline simultaneously

Better GPU utilization than naive model parallelism

Tensor Parallelism

Split individual layers across GPUs

Matrix multiply distributed (Megatron-LM)

High communication cost but good for huge models

Training and Optimization

Q6: How do you handle overfitting in deep learning?

Example

Detection: Training loss ↓ but validation loss ↑ Solutions (in order of typical application): 1. More data (always best if available) 2. Data augmentation (free additional data) 3. Dropout (0.1-0.5, typically 0.1 for transformers) 4. Weight decay (L2 regularization, 0.01 for AdamW) 5. Early stopping (monitor validation loss) 6. Reduce model size (fewer layers/neurons) 7. Label smoothing (0.1: softens hard targets) 8. Mixup/CutMix (interpolate between training examples)

Q7: Explain learning rate warmup and why it's used.

Warmup	Start with very small lr, linearly increase for N steps
Step 1-1000	lr = step/1000 × target_lr
Step 1000+	lr follows decay schedule (cosine, linear)
Without warmup: loss spike	NaN → training fails
With warmup: smooth start	stable optimization

Q8: What is mixed precision training?

Use FP16 (half precision) for most computation

- 2× faster matrix multiply on GPU

- 2× less memory → larger batch sizes

But keep FP32 for critical operations

- Loss computation (avoid overflow)

- Weight updates (small gradients need precision)

- Batch norm statistics

Loss scaling: Multiply loss by 1024 before backward pass

Prevents small gradients from underflowing in FP16

Divide gradients by 1024 before weight update

Result: Same accuracy as FP32, 2× training speed

Standard practice for all modern training

Practical Deep Learning

Q9: How do you debug a neural network that won't train?

1. Verify data	Are labels correct? Shuffle working?
2. Overfit one batch: If can't memorize 1 batch	bug in model
3. Check gradients	Are they flowing? (print gradient norms)
4. Simplify model	Does a tiny model learn anything?
5. Check learning rate	Try 10× smaller and 10× larger
6. Verify loss function	Does it match your task correctly?
7. Check initialization	Random weights in reasonable range?
8. Monitor activations	Are layers outputting constant values?
9. Check data pipeline	Is preprocessing correct?
10. Compare with known good	Does published code work?

Q10: How would you deploy a deep learning model in production?

Pipeline

Training: GPU cluster, large batches, days of training

Optimization: Quantization (INT8), pruning, distillation

Serving: TorchServe, TensorRT, ONNX Runtime

Monitoring: Track latency, accuracy drift, input distribution

Key considerations

- Latency requirements (real-time: <50ms, batch: seconds)

- Hardware (GPU server, edge device, mobile phone)

- Model size (edge: <50MB, server: unlimited)

- Batching strategy (dynamic batching for throughput)

- A/B testing (gradual rollout, compare with baseline)

- Retraining schedule (data drift → model staleness)

Interview Tips for Deep Learning Roles

Example

1. Know fundamentals deeply (backprop, attention, optimization) 2. Be ready to derive equations on whiteboard 3. Have 2-3 projects you can discuss in depth 4. Understand engineering aspects (deployment, scaling) 5. Know recent papers in your area (last 6 months) 6. Practice coding neural networks from scratch 7. Understand failure modes and debugging strategies

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for Deep Learning Interview Questions — Artificial Intelligence.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, deep, learning

Continue learning this concept

Interview PreparationML Interview QuestionsComplete ML interview prep. Algorithms, optimization. Interview 2024.Interview PreparationAI Interview QuestionsMaster AI interview prep. Algorithms, deep learning. Interview 2024.Interview PreparationLLM & RAG Interview QuestionsLearn LLM interview prep. Language models, RAG. Interview 2024.Interview PreparationAI Interview CheatsheetQuick reference for AI interviews. Algorithms, formulas. Interview 2024.Deep LearningBERT - Bidirectional Encoder RepresentationsMaster BERT architecture for AI. Learn bidirectional transformers, pretraining, fine-tuning, token embeddings, attention. Interview preparation 2024.Deep LearningConvolutional Neural Networks (CNN) - Complete AI GuideMaster CNN architecture for artificial intelligence. Learn convolution operation, pooling, filters, image classification, PyTorch implementation. Technical interview 2024.

PreviousAI Interview Questions NextLLM & RAG Interview Questions

AI Notes

Deep Learning Interview Questions — Artificial Intelligence

Master DL interview prep. Networks, training. Interview 2024.

Architecture Questions

Q1: Explain the Transformer architecture in detail.

Components

Encoder: Self-attention + FFN, bidirectional

Decoder: Masked self-attention + cross-attention + FFN

Self-Attention

Q, K, V = input × W_Q, W_K, W_V

Attention = softmax(QK^T/√d_k) × V

Multi-head: h parallel attention functions (h=8 typically)

Each head: d_k = d_model/h (64 for d_model=512)

Concat outputs → linear projection

FFN: Two-layer MLP applied per position

FFN(x) = ReLU(xW₁ + b₁)W₂ + b₂

Inner dimension 4× model dimension (2048 for d_model=512)

Residual connections + LayerNorm after each sublayer

Positional encoding: sinusoidal or learned

Q2: What are the key differences between BERT, GPT, and T5?

BERT (Encoder-only)

Bidirectional attention (sees full context)

Pre-training: MLM + NSP

Best for: Classification, NER, QA (understanding)

Cannot generate text naturally

GPT (Decoder-only)

Causal (left-to-right) attention

Pre-training: Next token prediction

Best for: Text generation, dialogue, code

Can do classification via prompting

T5 (Encoder-Decoder)

Full transformer (encoder + decoder)

Text-to-text framework (everything is seq2seq)

Best for: Translation, summarization, multi-task

"Translate English to French: ..." → "..."

Q3: How does batch normalization work and why is it useful?

For each mini-batch

μ_B = mean(batch)

σ²_B = variance(batch)

x̂ = (x - μ_B) / √(σ²_B + ε)

output = γx̂ + β (learned scale and shift)

Benefits

- Stabilizes training (prevents internal covariate shift)

- Allows higher learning rates (faster training)

- Acts as regularizer (batch statistics add noise)

vs. Layer Normalization:

BN: normalize across batch (good for CNNs)

LN: normalize across features (good for Transformers, RNNs)

LN doesn't depend on batch size → works for any batch/sequence

Q4: Explain residual connections and why they help.

Residual	output = F(x) + x (skip connection)
Without residual	network must learn identity if layer is useless
With residual	if F(x)=0 is best, output = 0 + x = x automatically
Used in	ResNets, Transformers, any deep architecture

Q5: What is the difference between model parallelism and data parallelism?

Data Parallelism

Same model copied to N GPUs

Each GPU processes different batch of data

Gradients averaged across GPUs, then applied

Scales to: large batches

Limitation: model must fit on one GPU

Model Parallelism

Model split across N GPUs

Each GPU holds different layers/parts

Data passes through GPUs sequentially

Scales to: very large models (GPT-3: 175B params)

Limitation: sequential dependency, GPU utilization

Pipeline Parallelism

Model split into stages across GPUs

Multiple mini-batches in pipeline simultaneously

Better GPU utilization than naive model parallelism

Tensor Parallelism

Split individual layers across GPUs

Matrix multiply distributed (Megatron-LM)

High communication cost but good for huge models

Training and Optimization

Q6: How do you handle overfitting in deep learning?

Example

Q7: Explain learning rate warmup and why it's used.

Warmup	Start with very small lr, linearly increase for N steps
Step 1-1000	lr = step/1000 × target_lr
Step 1000+	lr follows decay schedule (cosine, linear)
Without warmup: loss spike	NaN → training fails
With warmup: smooth start	stable optimization

Q8: What is mixed precision training?

Use FP16 (half precision) for most computation

- 2× faster matrix multiply on GPU

- 2× less memory → larger batch sizes

But keep FP32 for critical operations

- Loss computation (avoid overflow)

- Weight updates (small gradients need precision)

- Batch norm statistics

Loss scaling: Multiply loss by 1024 before backward pass

Prevents small gradients from underflowing in FP16

Divide gradients by 1024 before weight update

Result: Same accuracy as FP32, 2× training speed

Standard practice for all modern training

Practical Deep Learning

Q9: How do you debug a neural network that won't train?

1. Verify data	Are labels correct? Shuffle working?
2. Overfit one batch: If can't memorize 1 batch	bug in model
3. Check gradients	Are they flowing? (print gradient norms)
4. Simplify model	Does a tiny model learn anything?
5. Check learning rate	Try 10× smaller and 10× larger
6. Verify loss function	Does it match your task correctly?
7. Check initialization	Random weights in reasonable range?
8. Monitor activations	Are layers outputting constant values?
9. Check data pipeline	Is preprocessing correct?
10. Compare with known good	Does published code work?

Q10: How would you deploy a deep learning model in production?

Pipeline

Training: GPU cluster, large batches, days of training

Optimization: Quantization (INT8), pruning, distillation

Serving: TorchServe, TensorRT, ONNX Runtime

Monitoring: Track latency, accuracy drift, input distribution

Key considerations

- Latency requirements (real-time: <50ms, batch: seconds)

- Hardware (GPU server, edge device, mobile phone)

- Model size (edge: <50MB, server: unlimited)

- Batching strategy (dynamic batching for throughput)

- A/B testing (gradual rollout, compare with baseline)

- Retraining schedule (data drift → model staleness)

Interview Tips for Deep Learning Roles

Example

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for Deep Learning Interview Questions — Artificial Intelligence.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, deep, learning

Continue learning this concept

PreviousAI Interview Questions NextLLM & RAG Interview Questions