AI Notes

LLM & RAG Interview Questions

Learn LLM interview prep. Language models, RAG. Interview 2024.

Large Language Model Questions

Q1: How does GPT generate text?

Autoregressive generation

Input: "The cat sat on"

Model outputs probability distribution over vocabulary

Sample next token: "the" (P=0.3)

Append: "The cat sat on the"

Repeat until <EOS> or max length

Key details

- Causal attention mask prevents seeing future tokens

- Temperature controls randomness (low=deterministic, high=creative)

- Top-k/Top-p sampling prevents low-probability nonsense

- KV-cache stores previous computations for efficiency

Q2: What is the difference between fine-tuning and prompting?

Fine-tuning

Modify model weights on task-specific data

Requires: labeled dataset (1000+ examples)

Cost: GPU hours + data collection

Result: Specialized model, high accuracy on that task

Limitation: Catastrophic forgetting of other abilities

Prompting (in-context learning)

Give examples/instructions in the input

Requires: Good prompt design (no training data needed)

Cost: Only inference cost

Result: General model, decent accuracy on many tasks

Limitation: Limited by context window, inconsistent

Parameter-Efficient Fine-Tuning (PEFT)

LoRA: Add small trainable matrices (0.1% of params)

Prefix tuning: Prepend learnable tokens

Best of both: Cheap to train, preserves general knowledge

Q3: Explain RLHF (Reinforcement Learning from Human Feedback).

Stage 1	Supervised Fine-Tuning (SFT)
Result	Model follows instructions but imperfectly
Stage 2	Reward Model Training
Humans rank responses	A > B > C > D
RM(prompt, response)	scalar quality score
Stage 3	RL Optimization (PPO)
Result	ChatGPT-like models that are helpful, harmless, honest

Q4: What causes hallucination in LLMs and how to mitigate it?

Causes

- Training on incorrect data (internet has errors)

- Distributional patterns override factual recall

- No grounding in external truth (pattern matching only)

- Pressure to generate fluent text (confidence without knowledge)

Mitigation

1. RAG (Retrieval Augmented Generation): Ground in documents

2. Fine-tuning on verified facts

3. Training to say "I don't know" (calibration)

4. Chain-of-thought: Forces step-by-step reasoning

5. Fact-checking layer: Verify claims against KB

6. Citation requirement: Must cite source for claims

7. Lower temperature: Reduce creativity, increase factuality

Q5: How do you evaluate LLM quality?

Automated metrics

Perplexity: How well model predicts held-out text

BLEU/ROUGE: N-gram overlap for generation tasks

Accuracy: On benchmarks (MMLU, HellaSwag, GSM8K)

Human evaluation

Helpfulness: Does it answer the question?

Correctness: Are facts accurate?

Harmlessness: Is output safe and appropriate?

Coherence: Is it well-structured and logical?

LLM-as-judge

Use GPT-4 to evaluate other models' outputs

Cheap approximation of human eval

Correlates ~80% with human preferences

RAG (Retrieval-Augmented Generation)

Q6: Explain the RAG architecture.

Standard RAG pipeline

Query: "What is the capital of France?"

↓

[1. RETRIEVAL]

Embed query → vector search against document embeddings

Retrieve top-k relevant chunks (k=3-10)

[2. AUGMENTATION]

Prompt = "Context: {retrieved_chunks}\nQuestion: {query}\nAnswer:"

[3. GENERATION]

LLM generates answer grounded in retrieved context

→ "Based on the provided context, the capital of France is Paris."

Benefits over plain LLM

- Reduces hallucination (grounded in sources)

- Up-to-date information (documents can be refreshed)

- Attributable (can cite source documents)

- Domain-specific (knowledge base is customizable)

Q7: How do you chunk documents for RAG?

Chunking strategies

Fixed-size: Split every 500 tokens (with overlap)

Simple but may split mid-sentence/concept

Sentence-based: Split at sentence boundaries

Respects linguistic units

Semantic: Split at paragraph/section boundaries

Best preserves meaning

Recursive: Try large chunks first, split if too big

LangChain's RecursiveCharacterTextSplitter

Key parameters

Chunk size: 200-1000 tokens (larger = more context, less precise)

Overlap: 50-200 tokens (prevents information loss at boundaries)

Best practice: chunk_size=500, overlap=100 is a good starting point

Tune based on retrieval evaluation metrics

Q8: How do you evaluate a RAG system?

Retrieval evaluation

Recall@k: Is the answer in top-k retrieved chunks?

MRR: Mean Reciprocal Rank of correct document

nDCG: Normalized Discounted Cumulative Gain

Generation evaluation

Faithfulness: Does answer match retrieved context?

Answer relevancy: Does answer address the question?

Context precision: Are retrieved chunks relevant?

End-to-end

RAGAS framework: automated RAG evaluation

Human evaluation: correctness + completeness + citation accuracy

Common failure modes

1. Retrieval miss: Correct document not retrieved

2. Context too long: LLM ignores relevant part (lost in middle)

3. Extraction error: LLM misinterprets retrieved text

4. Query ambiguity: Retriever returns wrong topic's documents

Q9: What are embedding models and how to choose one?

Embedding models convert text	dense vectors for similarity search
OpenAI text-embedding-3-large	3072 dims, excellent quality
Cohere embed-v3	Multilingual, good compression
BGE/E5	Open-source, competitive quality
all-MiniLM-L6	Fast, lightweight (384 dims)
- Quality	MTEB benchmark scores
- Dimensionality	Higher = more precise, more storage
- Speed	Inference latency for your volume
- Cost	API pricing or self-hosting compute
- Domain	Some specialized for code, medical, legal
Cosine similarity	Most common, normalized
Dot product	Faster, requires normalized vectors
Euclidean distance	When magnitude matters

Q10: How do you handle RAG at scale?

Vector database options

Pinecone: Managed, auto-scaling

Weaviate: Open-source, hybrid search

Qdrant: High performance, filtering

pgvector: PostgreSQL extension (familiar)

FAISS: Library (not database), fastest for research

Scaling strategies

- Index sharding: Split vectors across multiple indices

- Approximate NN: HNSW/IVF trade accuracy for speed

- Metadata filtering: Pre-filter before vector search

- Hybrid search: BM25 keyword + vector semantic search

- Caching: Cache frequent query results

Production considerations

- Index refresh: How often to re-embed new documents?

- Versioning: Track which index version served which results

- Monitoring: Track retrieval quality degradation

- Fallback: What if retrieval returns nothing relevant?

Interview Tips

Example

For LLM/RAG roles: - Understand the full pipeline (embedding → retrieval → generation) - Know trade-offs between approaches (fine-tuning vs RAG vs prompting) - Have hands-on experience with at least one vector database - Understand evaluation (automated + human) - Know failure modes and how to diagnose/fix them - Stay current (field moves fast — read papers weekly)

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for LLM & RAG Interview Questions.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.

Search Terms

artificial-intelligence, artificial intelligence, artificial, intelligence, interview, preparation, llm, and

Continue learning this concept

Interview PreparationAI Interview QuestionsMaster AI interview prep. Algorithms, deep learning. Interview 2024.Interview PreparationDeep Learning Interview Questions — Artificial IntelligenceMaster DL interview prep. Networks, training. Interview 2024.Interview PreparationML Interview QuestionsComplete ML interview prep. Algorithms, optimization. Interview 2024.Interview PreparationAI Interview CheatsheetQuick reference for AI interviews. Algorithms, formulas. Interview 2024.Ai Agents And LlmsRAG: Retrieval-Augmented GenerationImplement RAG systems that ground LLM responses in external knowledge.Ai Agents And LlmsAgentic AI ProjectsBuild practical agentic AI systems with multi-tool orchestration, autonomous planning, and real-world applications.

PreviousDeep Learning Interview Questions — Artificial Intelligence NextML Interview Questions