AI Notes
Master NLP basics. Text processing, tokenization, embeddings. BTech guide 2024.
What is NLP?
Natural Language Processing (NLP) is the branch of artificial intelligence focused on enabling computers to understand, interpret, generate, and meaningfully respond to human language. It bridges the gap between human communication and computational processing, transforming unstructured text into structured information that machines can act upon.
NLP is everywhere: when you ask Siri a question, use Google Translate, receive email spam filtering, get autocomplete suggestions, or interact with a chatbot—NLP is working behind the scenes. The field combines linguistics, computer science, and statistical methods to tackle one of AI's hardest challenges: understanding the ambiguity, context-dependence, and creativity of human language.
Why is NLP Hard?
Human language presents unique challenges that make NLP fundamentally difficult:
Ambiguity at Every Level
Lexical ambiguity (word level)
"bank" → financial institution OR river edge
"bat" → flying animal OR sports equipment
Syntactic ambiguity (structure level)
"I saw the man with the telescope"
→ I used a telescope to see him? OR He had a telescope?
Semantic ambiguity (meaning level)
"Every student read a book"
→ All read the same book? OR Each read a different book?
Pragmatic ambiguity (intent level)
"Can you pass the salt?"
→ Question about ability? OR request to pass salt?
Context Dependence
NLP Pipeline: Traditional Approach
| Raw Text | Tokenization → POS Tagging → Parsing → Semantics → Application |
| Step 1 | TOKENIZATION |
| "I can't believe it!" | ["I", "ca", "n't", "believe", "it", "!"] |
| Step 2 | PART-OF-SPEECH TAGGING |
| Step 3 | SYNTACTIC PARSING |
| Step 4 | NAMED ENTITY RECOGNITION |
| Step 5 | SEMANTIC ANALYSIS |
| Intent | denial, Object: unspecified event |
Core NLP Tasks
Text Classification
| Input | Document/sentence |
| Output | Category label |
| - Spam detection: email | spam/not_spam |
| - Sentiment analysis: review | positive/negative |
| - Topic classification: article | politics/sports/tech |
| - Language detection: text | english/spanish/chinese |
Information Extraction
| Input | Unstructured text |
| Output | Structured data |
| - Named Entity Recognition | find people, places, dates |
| - Relation Extraction: "Einstein born in Ulm" | (Einstein, birthPlace, Ulm) |
| - Event Extraction | who did what to whom when where |
Text Generation
| Input | Prompt, context, or structured data |
| Output | Human-readable text |
| - Machine translation: English | French |
| - Summarization: long article | key points |
| - Dialogue systems: user query | response |
| - Creative writing: prompts | stories |
Question Answering
| Input | Question + context (or knowledge base) |
| Output | Answer |
| - Extractive QA | answer is a span in the context |
| - Generative QA | answer is generated from knowledge |
| - Open-domain | answer from large corpus (Google, ChatGPT) |
Evolution of NLP Approaches
Rule-Based Era (1950s-1980s)
| ELIZA (1966) | Pattern matching chatbot |
| Input | "I am sad" |
| Pattern: "I am X" | Response: "How long have you been X?" |
| Pros | Precise for narrow domains |
| Cons | Doesn't scale, misses nuance |
Statistical Era (1990s-2010s)
| Algorithms | Naive Bayes, SVM, HMM, CRF |
| - Punctuation (!!! | strong emotion) |
| Pros | Data-driven, handles variation |
| Cons | Requires manual feature engineering |
Deep Learning Era (2013-2017)
| Word2Vec (2013) | Distributed word representations |
| Pros | No feature engineering, better accuracy |
| Cons | Needs lots of labeled data per task |
Pre-trained Language Model Era (2018-present)
| BERT (2018) | Bidirectional understanding |
| GPT-3 (2020) | Few-shot generation |
| ChatGPT (2022) | Instruction following |
| Paradigm | Pre-train once, fine-tune/prompt for any task |
| Pros | State-of-the-art everything, few-shot learning |
| Cons | Compute intensive, potential biases, hallucination |
Word Representations
One-Hot Encoding (Simplest)
Word Embeddings (Dense Vectors)
Evaluation Metrics
| Classification | Accuracy, Precision, Recall, F1-score |
| Generation | BLEU (translation), ROUGE (summarization) |
| QA | Exact Match, F1 over tokens |
| Perplexity | How well model predicts next word (lower = better) |
| Human evaluation | Fluency, adequacy, coherence (gold standard) |
Interview Questions
Q: Why is NLP considered harder than computer vision? A: Language is inherently ambiguous, context-dependent, and creative. Two images of cats look similar; two sentences meaning the same thing can look completely different ("The cat sat" vs "A feline positioned itself"). Also, language understanding requires world knowledge, cultural context, and theory of mind.
Q: What changed with transformers/BERT? A: The paradigm shifted from training separate models per task to pre-training one large model on raw text, then adapting it. This solved the data scarcity problem—you only need thousands (not millions) of labeled examples for fine-tuning, because the model already understands language from pre-training.
Q: What is the difference between NLP and NLU? A: NLP is the broad field encompassing all computational work with language. NLU (Natural Language Understanding) specifically focuses on comprehension—extracting meaning and intent. NLG (Natural Language Generation) focuses on producing language. NLP = NLU + NLG + processing tasks like tokenization.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for NLP Fundamentals - Complete Guide.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.
Search Terms
artificial-intelligence, artificial intelligence, artificial, intelligence, natural, language, processing, nlp
Related Artificial Intelligence Topics