Python Notes
Master PDF automation with Python using PyPDF2, reportlab, pdfplumber for reading, writing, merging, splitting PDFs, extracting text/tables, and generating PDF reports.
Introduction
You can read, write, merge, split, and generate PDFs using Python. Three main libraries are used:
- PyPDF2 — read, merge, split, rotate, encrypt
- pdfplumber — accurate text aur table extraction
- reportlab — professional PDF generation
pip install pypdf2 pdfplumber reportlab pillow2. Merging and Splitting PDFs
Adding: /tmp/.../part1.pdf Adding: /tmp/.../part2.pdf Merged 2 PDFs -> /tmp/.../merged.pdf Merged PDF pages: 2
3. Rotation, Watermark, Encryption
Watermarked PDF saved to output_watermarked.pdf Encrypted PDF saved to output_encrypted.pdf Decrypted successfully!
4. pdfplumber — Accurate Text Extraction
Total pages: 3
Metadata: {'Title': 'Invoice Report', 'Author': 'WoHoTech'}
Page 1, Table 1: 5 rows
Page 2, Table 1: 8 rows
Page 2, Table 2: 3 rows📝 Hindi Explanation
pdfplumber provides more accurate text extraction than PyPDF2, especially for tables. extract_tables() automatically detects table structure and returns data as a list of lists.5. ReportLab — Generate Professional PDFs
Report generated: /tmp/.../student_report.pdf Report size: 4523 bytes
6. Advanced ReportLab — Charts and Images
Bar chart created: Subject Scores Pie chart created: Score Distribution Analytics report saved to analytics_report.pdf
7. Best Practices
import PyPDF2
from pathlib import Path
# ✅ Always use context managers
with open("file.pdf", "rb") as f:
reader = PyPDF2.PdfReader(f)
# ✅ Handle encrypted PDFs
with open("protected.pdf", "rb") as f:
reader = PyPDF2.PdfReader(f)
if reader.is_encrypted:
reader.decrypt("password")
# ✅ Check file existence
def safe_read_pdf(filepath):
if not Path(filepath).exists():
raise FileNotFoundError(f"PDF not found: {filepath}")
if Path(filepath).suffix.lower() != ".pdf":
raise ValueError("Not a PDF file!")
with open(filepath, "rb") as f:
return PyPDF2.PdfReader(f)
# ✅ pdfplumber for text extraction (better than PyPDF2)
import pdfplumber
with pdfplumber.open("document.pdf") as pdf:
text = "\n".join(page.extract_text() or "" for page in pdf.pages)
# ✅ ReportLab for generating (don't edit existing PDFs)| Library | Best For |
|---|---|
PyPDF2 | Merge, split, rotate, encrypt |
pdfplumber | Text/table extraction |
reportlab | Generate new PDFs |
pdf2image | PDF to images |
fpdf2 | Simple PDF generation |
Pro Tip: PyPDF2 and pdfplumber are for text extraction — PyPDF2 is fast, pdfplumber is more accurate. For complex tables, ALWAYS use pdfplumber!
⚠️ Common Mistakes
❌ Mistake 1: Opening a file in text mode
# ❌ Wrong — PDF binary file hai, "r" mode se error aayega
with open("file.pdf", "r") as f:
reader = PyPDF2.PdfReader(f)
# ✅ Correct — Always use "rb" (read binary)
with open("file.pdf", "rb") as f:
reader = PyPDF2.PdfReader(f)PDF is a binary format; using"r"mode will give youUnicodeDecodeErroror corrupted data. Always use"rb"! 🔑
❌ Mistake 2: Reading an encrypted PDF without decrypting
You must calldecrypt()before performing any operation on an encrypted PDF, otherwise you'll getPdfReadError. 🔐
❌ Mistake 3: Not handling None return from extract_text()
# ❌ Wrong — None pe string operations crash karengi
text = page.extract_text()
words = text.split() # TypeError if text is None!
# ✅ Correct — None check karo
text = page.extract_text() or ""
words = text.split()Some pages contain only images, no text —extract_text()will returnNone. Always provide a fallback! ⚡
❌ Mistake 4: Loading a large PDF entirely into memory without streaming
# ❌ Wrong — puri PDF memory mein load for huge files
all_text = ""
for page in reader.pages:
all_text += page.extract_text() or "" # Memory blast! 💥
# ✅ Correct — page by page process karo
for page_num, page in enumerate(reader.pages):
text = page.extract_text() or ""
process_page(text) # Ek page process, fir nextStoring all content from a 1000+ page PDF in one string will cause memory overflow. Use page-by-page processing! 💾
❌ Mistake 5: Not closing the file in ReportLab
# ❌ Wrong — canvas.save() bhoolna
from reportlab.pdfgen import canvas
c = canvas.Canvas("output.pdf")
c.drawString(100, 750, "Hello")
# File is incomplete without save()!
# ✅ Correct — always call save()
c = canvas.Canvas("output.pdf")
c.drawString(100, 750, "Hello")
c.save() # File finalize hota haiWithout canvas.save(), the PDF will remain corrupt. This properly closes and finalizes the file! 📄❌ Mistake 6: Extracting complex tables with PyPDF2
PyPDF2 only gives raw text; table layout is lost. For tables, usepdfplumberorcamelot! 📊
❌ Mistake 7: PdfMerger close na karna
# ❌ Wrong — merger.close() bhoolna
merger = PyPDF2.PdfMerger()
merger.append("file1.pdf")
merger.append("file2.pdf")
merger.write(open("output.pdf", "wb"))
# File handles leak ho jayenge!
# ✅ Correct — proper close karo
merger = PyPDF2.PdfMerger()
merger.append("file1.pdf")
merger.append("file2.pdf")
with open("output.pdf", "wb") as f:
merger.write(f)
merger.close() # Resources free!PdfMergerinternally keeps file handles open. Not callingclose()causes file locks and memory leaks! 🚫
✅ Key Takeaways
- 📌 PyPDF2 is best for PDF merge, split, rotate, and encrypt — limited in text extraction
- 📌 pdfplumber is the gold standard for accurate text and table extraction — especially for invoices and reports
- 📌 reportlab lets you generate professional PDFs with charts, tables, images, and custom styles
- 📌 Always open PDF files in binary mode (
"rb") — they are not text files - 📌 Always call
decrypt()before working with encrypted PDFs - 📌
extract_text()can returnNone— always use anor ""fallback - 📌 For large PDFs, use page-by-page processing — don't load the entire file into memory at once
- 📌 In ReportLab, calling
canvas.save()ordoc.build(story)is required — without it the PDF will be corrupt - 📌 Avoid PyPDF2 for table extraction — use
pdfplumberorcamelotinstead - 📌 Error handling is important in PDF automation — handle file not found, wrong password, and corrupted PDF scenarios
❓ FAQ
Q1: What is the difference between PyPDF2 and pdfplumber? 🤔
Answer: PyPDF2 is mainly for PDF manipulation (merge, split, rotate, encrypt). It can extract text but with limited accuracy. pdfplumber specializes in accurate text and table extraction — it understands page layout and coordinates.
Q2: Can you edit an existing PDF with Python? ✏️
Answer: Not directly — PDF format is not designed for editing. However, workarounds exist: (1) Extract text, modify it, generate a new PDF, (2) Use PyPDF2 to add watermarks/overlays, (3) Use pdf2docx to convert, edit in Word format, then convert back.
Q3: How do you extract images from a PDF? 🖼️
Answer: Use the pdf2image library which converts pages to images. After pip install pdf2image, use convert_from_path("file.pdf") to get PIL Image objects. For embedded images specifically, use fitz (PyMuPDF) library.
Q4: How do you add Hindi/Unicode text in ReportLab? 🇮🇳
Answer: For Unicode support in ReportLab, register a TTF font: pdfmetrics.registerFont(TTFont('Hindi', 'path/to/font.ttf')) and then use that font name in your styles/canvas.
Q5: How do you efficiently process a very large PDF (1000+ pages)? ⚡
Answer: Iterate page-by-page; don't keep everything in memory. Use a generator pattern. pdfplumber already uses lazy loading. If you only need specific pages, pass page numbers directly to avoid processing the entire document.
Q6: What are the best practices for PDF generation? 📋
Answer: (1) Use SimpleDocTemplate for complex layouts with Platypus flowables, (2) Define custom ParagraphStyle for reusability, (3) Use page margins and headers/footers for professional output, (4) Always test with various PDF viewers.
Q7: How do you extract text from scanned PDFs? 📸
Answer: Scanned PDFs contain text as images, so normal extraction won't work. You need OCR: use the pytesseract + pdf2image combination. Convert each page to an image, then apply OCR. For better accuracy, preprocess images (denoise, deskew).
Q8: How do you automate multiple PDF operations? 🤖
Answer: Use pathlib or glob in a Python script to iterate over all PDFs in a folder and apply operations (rename, merge, extract). Use the watchdog library to automatically trigger processing when new PDFs appear in a folder.
🎯 Interview Questions
Q1: What is the fundamental difference between PyPDF2, pdfplumber, and reportlab?
Answer: These three libraries serve different purposes: PyPDF2 is a PDF manipulation library — merge, split, rotate, encrypt/decrypt, read metadata. pdfplumber specializes in extraction — accurate text and table extraction with coordinate awareness. reportlab is for creation — generate new PDFs with charts, tables, custom layouts from scratch.
Q2: How does text extraction from PDFs work internally?
Answer: PDF internally stores text in "content streams" with positioning operators (Tm, Td, etc.). The extraction process: (1) Parse PDF structure, (2) Read content streams, (3) Map character codes to Unicode via CMap/Encoding, (4) Use positioning info to reconstruct text flow. That's why extraction isn't always perfect.
Q3: How do you manage memory when processing large PDF files?
Answer: Strategies: (1) Lazy iteration — PdfReader.pages is a list, but page content loads only when accessed, (2) Generator pattern — yield one page at a time to keep memory constant, (3) Process in chunks — split large PDFs first, process chunks independently.
Q4: PDF encryption levels explain karo — user password vs owner password?
Answer: PDF has two levels of encryption: User password — required to open the PDF; without it the PDF won't display. Owner password — restricts operations (print, copy, edit); the PDF opens but with limitations. Python can handle both via decrypt(password).
Q5: How does pdfplumber detect tables?
Answer: pdfplumber's table detection is a multi-step process: (1) Line detection — find horizontal and vertical lines on the page (explicit drawn lines), (2) Edge inference — where lines aren't drawn, infer edges from text alignment, (3) Cell identification — intersections of lines form cells, (4) Content extraction — text within each cell boundary.
Q6: What is the Platypus framework in ReportLab and when should you use it?
Answer: Platypus (Page Layout and Typography Using Scripts) is reportlab's high-level framework. It uses "flowable" objects — Paragraph, Table, Image, Spacer — that automatically flow across pages. Use it for multi-page documents; use Canvas for single-page or precise positioning needs.
Q7: What are the best practices for error handling in PDF automation?
Answer: Handle critical errors: (1) FileNotFoundError — check if file exists before opening, (2) PdfReadError — corrupted PDF, wrap in try-except, (3) PermissionError — file locked by another process, (4) Password errors — wrong decryption password. Always provide meaningful error messages and fallbacks.
Q8: Design a batch PDF processing system — you need to extract data from 1000 invoices.
Answer: Architecture: (1) Input — receive PDF paths from folder watch or queue, (2) Validation — check file exists, is PDF, not corrupted, size check, (3) Extraction — pdfplumber for tables, targeted area cropping for specific fields, (4) Parsing — regex/rules to structure raw text, (5) Storage — database insert with error logging, (6) Reporting — success/failure stats, problematic files flagged for manual review.
Q9: Explain the workflow of extracting structured data from PDFs and storing it in a database.
Answer: Step-by-step: (1) Open PDF with pdfplumber, (2) Use page.crop() to target relevant areas — header, line items, totals, (3) Use extract_text() for simple fields and extract_tables() for tabular data, (4) Parse and validate with regex/rules, (5) Map to database schema, (6) Insert with error handling and transaction management.
Q10: What are the limitations of Python PDF libraries and what are the alternatives?
Answer: Limitations: (1) PyPDF2 — wrong text order in complex layouts, no scanned PDF support, (2) pdfplumber — slow on very large files, no image extraction, (3) reportlab — steep learning curve, no PDF editing. Alternatives: Apache PDFBox (Java, very robust), MuPDF/PyMuPDF (fast, feature-rich), pdf2docx (editing workflow), Adobe PDF Services API (cloud-based, most accurate).
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for PDF Automation with Python.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, automation, pdf, pdf automation with python
Related Python Master Course Topics