AI Notes

Build AI Chatbot

Create chatbot system. NLP, intent recognition. Hands-on 2024.

Project Overview

Build an intelligent chatbot that can understand user intent, maintain conversation context, retrieve relevant information, and generate helpful responses. This project progresses from rule-based pattern matching through retrieval-based approaches to modern transformer-powered generation, teaching the full spectrum of conversational AI techniques.

A chatbot combines multiple AI disciplines: natural language understanding (parsing intent from free text), information retrieval (finding relevant knowledge), dialogue management (tracking conversation state), and natural language generation (producing fluent responses). Building one end-to-end gives you practical experience with the entire NLP pipeline.

Architecture Options

Level 1: Rule-Based Chatbot

Example

import re, random class RuleBasedBot: def __init__(self): self.rules = [ (r'hi|hello|hey', self.greet), (r'how are you', self.status), (r'what is (.*)', self.define), (r'help|what can you do', self.capabilities), (r'bye|goodbye|quit', self.farewell), ] self.knowledge = { 'python': 'Python is a high-level programming language.', 'ai': 'AI is the simulation of human intelligence by machines.', 'machine learning': 'ML is a subset of AI that learns from data.', } def respond(self, user_input): for pattern, handler in self.rules: match = re.search(pattern, user_input.lower()) if match: return handler(match) return "I'm not sure I understand. Can you rephrase?" def greet(self, match): return random.choice(["Hello!", "Hi there!", "Hey! How can I help?"]) def define(self, match): term = match.group(1).strip().lower() return self.knowledge.get(term, f"I don't have info about '{term}' yet.") def capabilities(self, match): return "I can answer questions, have conversations, and help with information lookup." def farewell(self, match): return "Goodbye! Have a great day!"

Pros: Simple, fast, predictable, no training data needed Cons: Brittle, can't handle paraphrases, doesn't scale

Level 2: Intent Classification + Slot Filling

Example

from sklearn.pipeline import Pipeline from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.svm import LinearSVC class IntentBot: def __init__(self): self.intent_classifier = Pipeline([ ('tfidf', TfidfVectorizer(ngram_range=(1,2))), ('clf', LinearSVC()) ]) self.slot_extractor = load_ner_model() self.dialogue_state = {} def train(self, training_data): texts = [d['text'] for d in training_data] intents = [d['intent'] for d in training_data] self.intent_classifier.fit(texts, intents) def process(self, user_input): intent = self.intent_classifier.predict([user_input])[0] entities = self.slot_extractor.extract(user_input) self.dialogue_state.update(entities) return self.fulfillment(intent, self.dialogue_state) def fulfillment(self, intent, state): if intent == 'book_flight': if 'destination' not in state: return "Where would you like to fly?" if 'date' not in state: return f"When would you like to fly to {state['destination']}?" return f"Searching flights to {state['destination']} on {state['date']}..." elif intent == 'get_weather': location = state.get('location', 'your area') return f"The weather in {location} is sunny, 72°F."

Level 3: RAG-Powered Chatbot

Example

from sentence_transformers import SentenceTransformer import faiss import numpy as np class RAGChatbot: def __init__(self, documents): self.embedder = SentenceTransformer('all-MiniLM-L6-v2') self.documents = documents self.index = self.build_index(documents) self.conversation_history = [] def build_index(self, docs): embeddings = self.embedder.encode(docs) index = faiss.IndexFlatL2(embeddings.shape[1]) index.add(embeddings.astype('float32')) return index def retrieve(self, query, k=3): query_vec = self.embedder.encode([query]).astype('float32') distances, indices = self.index.search(query_vec, k) return [self.documents[i] for i in indices[0]] def respond(self, user_query): # Retrieve relevant context relevant_docs = self.retrieve(user_query) context = "\n".join(relevant_docs) # Build prompt history = self.format_history() prompt = f"""Based on the following context, answer the user's question. If the context doesn't contain the answer, say so honestly. Context: {context} Conversation history: {history} User: {user_query} Assistant:""" # Generate response (using LLM API) response = llm_generate(prompt) # Update history self.conversation_history.append(('user', user_query)) self.conversation_history.append(('assistant', response)) return response def format_history(self, max_turns=5): recent = self.conversation_history[-max_turns*2:] return "\n".join([f"{role}: {msg}" for role, msg in recent])

Key Components

Conversation Memory Management

Example

class ConversationMemory: def __init__(self, max_turns=10): self.short_term = [] # Recent turns (full text) self.long_term = {} # Summarized facts about user self.max_turns = max_turns def add_turn(self, role, message): self.short_term.append({'role': role, 'content': message}) # Extract and store user preferences if role == 'user': self.extract_preferences(message) # Summarize if too long if len(self.short_term) > self.max_turns * 2: self.summarize_and_trim() def extract_preferences(self, message): # "I prefer Python over Java" # Store: {language_preference: "Python"} pass def summarize_and_trim(self): old_turns = self.short_term[:-self.max_turns] summary = llm_summarize(old_turns) self.long_term['conversation_summary'] = summary self.short_term = self.short_term[-self.max_turns:]

Error Handling and Fallbacks

python example

class FallbackHandler:
    def __init__(self):
        self.confidence_threshold = 0.5
        self.clarification_count = 0
    
    def handle(self, intent_confidence, user_input):
        if intent_confidence < self.confidence_threshold:
            self.clarification_count += 1
            if self.clarification_count >= 3:
                return "I'm having trouble understanding. Let me connect you with a human."
            return "I'm not quite sure what you mean. Could you rephrase that?"
        self.clarification_count = 0
        return None  # No fallback needed

Evaluation Framework

Automated Metrics

Intent accuracy: % correctly classified intents (target: >90%)

Entity extraction F1: slot filling accuracy (target: >85%)

Response relevance: BLEU/ROUGE against reference (baseline comparison)

Retrieval recall@k: correct doc in top-k retrieved

Human Evaluation

Relevance (1-5): Does the response address the question?

Fluency (1-5): Is the response natural and grammatical?

Helpfulness (1-5): Does it actually solve the user's problem?

Consistency: No contradictions within conversation? (binary)

User Metrics

Task completion rate: % of goals achieved

Conversation length: Shorter = more efficient

User satisfaction: Post-conversation survey

Return rate: Do users come back?

Project Milestones

Week 1-2	Rule-based prototype + CLI interface
Week 3-4	Intent classifier training (100+ examples per intent)
Week 5-6	RAG pipeline (chunk documents, build vector index)
Week 7-8	LLM integration + conversation memory
Week 9-10	Web UI (Streamlit/Gradio) + evaluation
Week 11-12	Deployment, monitoring, user testing

Technologies Stack

NLU	spaCy (NER), scikit-learn or fine-tuned BERT (intents)
Embeddings	sentence-transformers (all-MiniLM-L6-v2)
Vector DB	FAISS (local), ChromaDB, or Pinecone (cloud)
LLM	OpenAI GPT-3.5/4, Llama 2, Mistral (open-source)
Frontend	Streamlit (quick), Gradio (ML-focused), React (production)
Backend	FastAPI + WebSocket for streaming
Deployment	Docker + cloud (AWS/GCP) or local Raspberry Pi

Interview Questions

Q: How do you handle out-of-scope queries? A: Implement confidence thresholds on intent classification. If confidence < threshold, trigger fallback: acknowledge uncertainty, ask for clarification, or gracefully redirect. For RAG systems, if retrieved documents have low similarity scores (<0.3), respond "I don't have information about that" rather than generating from potentially irrelevant context. Track out-of-scope queries to expand coverage.

Q: How do you evaluate chatbot quality at scale? A: Combine automated metrics (latency, intent accuracy, retrieval relevance) with periodic human evaluation on sampled conversations. Track implicit satisfaction signals: conversation length, rephrasing rate (indicates misunderstanding), explicit feedback buttons, and task completion rate. Use LLM-as-judge (GPT-4 evaluating responses) for scalable quality assessment between expensive human evaluation rounds.

Q: How do you prevent hallucination in a RAG chatbot? A: Three layers of defense: (1) Retrieval quality—ensure retrieved documents are actually relevant (threshold on similarity score). (2) Prompt engineering—instruct the LLM to only answer from provided context and say "I don't know" otherwise. (3) Post-generation verification—check if the response is supported by the retrieved text. Log and review cases where the system says "I don't know" to identify knowledge gaps.

Continue the lesson path

Study next in this module

4 connected lessons

01ProjectsAI Agent ProjectBuild intelligent agent. Reinforcement learning, autonomy. Project 2024.02ProjectsFace Recognition System — Artificial IntelligenceBuild recognition system. Detection, verification. CV project 2024.03ProjectsPersonal LLM AssistantCreate LLM assistant. Fine-tuning, RAG. Project 2024.04ProjectsRecommendation System — Artificial IntelligenceCreate recommendation engine. Collaborative filtering. Project 2024.

PreviousAI Agent Project NextFace Recognition System — Artificial Intelligence

AI Notes

Build AI Chatbot

Create chatbot system. NLP, intent recognition. Hands-on 2024.

Project Overview

Architecture Options

Level 1: Rule-Based Chatbot

Example

Pros: Simple, fast, predictable, no training data needed Cons: Brittle, can't handle paraphrases, doesn't scale

Level 2: Intent Classification + Slot Filling

Example

Level 3: RAG-Powered Chatbot

Example

Key Components

Conversation Memory Management

Example

Error Handling and Fallbacks

python example

class FallbackHandler:
    def __init__(self):
        self.confidence_threshold = 0.5
        self.clarification_count = 0
    
    def handle(self, intent_confidence, user_input):
        if intent_confidence < self.confidence_threshold:
            self.clarification_count += 1
            if self.clarification_count >= 3:
                return "I'm having trouble understanding. Let me connect you with a human."
            return "I'm not quite sure what you mean. Could you rephrase that?"
        self.clarification_count = 0
        return None  # No fallback needed

Evaluation Framework

Automated Metrics

Intent accuracy: % correctly classified intents (target: >90%)

Entity extraction F1: slot filling accuracy (target: >85%)

Response relevance: BLEU/ROUGE against reference (baseline comparison)

Retrieval recall@k: correct doc in top-k retrieved

Human Evaluation

Relevance (1-5): Does the response address the question?

Fluency (1-5): Is the response natural and grammatical?

Helpfulness (1-5): Does it actually solve the user's problem?

Consistency: No contradictions within conversation? (binary)

User Metrics

Task completion rate: % of goals achieved

Conversation length: Shorter = more efficient

User satisfaction: Post-conversation survey

Return rate: Do users come back?

Project Milestones

Week 1-2	Rule-based prototype + CLI interface
Week 3-4	Intent classifier training (100+ examples per intent)
Week 5-6	RAG pipeline (chunk documents, build vector index)
Week 7-8	LLM integration + conversation memory
Week 9-10	Web UI (Streamlit/Gradio) + evaluation
Week 11-12	Deployment, monitoring, user testing

Technologies Stack

NLU	spaCy (NER), scikit-learn or fine-tuned BERT (intents)
Embeddings	sentence-transformers (all-MiniLM-L6-v2)
Vector DB	FAISS (local), ChromaDB, or Pinecone (cloud)
LLM	OpenAI GPT-3.5/4, Llama 2, Mistral (open-source)
Frontend	Streamlit (quick), Gradio (ML-focused), React (production)
Backend	FastAPI + WebSocket for streaming
Deployment	Docker + cloud (AWS/GCP) or local Raspberry Pi

Interview Questions

Continue the lesson path

Study next in this module

4 connected lessons

PreviousAI Agent Project NextFace Recognition System — Artificial Intelligence