Python Notes
Comprehensive guide to Python context managers using with statement, __enter__ and __exit__ methods, contextlib module, nested context managers and custom resource management.
Introduction
Context Managers are an elegant solution for resource management in Python. Using the with statement, you guarantee that resources will be properly opened and closed, regardless of whether an exception occurs or the code completes normally.
Real-world analogy: Turning off the light when leaving the house — no matter what happens, the light will be off on exit.
2. __enter__ and __exit__ Protocol
3. __exit__ Parameters Explained
class ExceptionHandler:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""
exc_type: Exception class (None agar koi exception nahi)
exc_val: Exception instance (None agar koi exception nahi)
exc_tb: Traceback object (None agar koi exception nahi)
"""
if exc_type is None:
print("No exception occurred")
return True
print(f"Exception type: {exc_type}")
print(f"Exception value: {exc_val}")
print(f"Traceback: {exc_tb}")
# Specific exceptions handle karo
if exc_type == ZeroDivisionError:
print("Handling ZeroDivisionError — suppressing it")
return True # Suppress the exception
return False # Other exceptions propagate karo
with ExceptionHandler():
x = 10 / 0 # ZeroDivisionError — suppressed!
print("After with block — no exception!")
with ExceptionHandler():
y = 5 / 2 # No exception
# Output: No exception occurred
try:
with ExceptionHandler():
raise TypeError("Type error") # Not suppressed
except TypeError:
print("TypeError propagated")4. contextlib.contextmanager
📝 Hindi Explanation
@contextmanagerdecorator lets you convert a generator function into a context manager. Code beforeyieldacts as__enter__, and code afteryieldacts as__exit__. This eliminates the need to write a full class.
5. Database Connection Context Manager
import sqlite3
from contextlib import contextmanager
class DatabaseManager:
"""Manage database connection and transactions"""
def __init__(self, db_path):
self.db_path = db_path
self.connection = None
def __enter__(self):
self.connection = sqlite3.connect(self.db_path)
self.connection.row_factory = sqlite3.Row
print(f"Connected to {self.db_path}")
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None:
print(f"Error occurred, rolling back: {exc_val}")
self.connection.rollback()
else:
print("Committing transaction")
self.connection.commit()
self.connection.close()
print("Connection closed")
return False # Don't suppress exceptions
# Usage
with DatabaseManager(":memory:") as conn:
cursor = conn.cursor()
cursor.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
cursor.execute("INSERT INTO users (name) VALUES (?)", ("Alice",))
cursor.execute("INSERT INTO users (name) VALUES (?)", ("Bob",))
cursor.execute("SELECT * FROM users")
for row in cursor.fetchall():
print(dict(row))
# Output:
# Connected to :memory:
# {'id': 1, 'name': 'Alice'}
# {'id': 2, 'name': 'Bob'}
# Committing transaction
# Connection closed6. contextlib Module Tools
from contextlib import suppress, redirect_stdout, redirect_stderr
import io
# suppress() — specific exceptions ignore karo
with suppress(FileNotFoundError):
with open("nonexistent.txt") as f:
print(f.read())
print("No exception raised!") # Program continues normally
# Equivalent to:
# try:
# with open("nonexistent.txt") as f: ...
# except FileNotFoundError:
# pass
# redirect_stdout — print output capture karo
output_buffer = io.StringIO()
with redirect_stdout(output_buffer):
print("This goes to buffer, not console!")
print("Me too!")
captured = output_buffer.getvalue()
print(f"Captured: {repr(captured)}")
# Output: Captured: 'This goes to buffer, not console!\nMe too!\n'
# redirect_stderr
error_buffer = io.StringIO()
with redirect_stderr(error_buffer):
import sys
print("Error message", file=sys.stderr)
print(f"Captured stderr: {repr(error_buffer.getvalue())}")7. ExitStack — Dynamic Context Managers
📝 Hindi Explanation
ExitStackis useful when you need to dynamically manage a variable number of resources. Eachenter_context()call adds a new resource, and on exit, all are closed in reverse order (LIFO).
8. Nested Context Managers
# Traditional nesting
with open("input.txt", "w") as infile:
infile.write("line1\nline2\nline3\n")
with open("input.txt", "r") as infile:
with open("output.txt", "w") as outfile:
for line in infile:
outfile.write(line.upper())
# Compact syntax (Python 3.1+)
with open("input.txt", "r") as infile, open("output.txt", "w") as outfile:
for line in infile:
outfile.write(line.upper())
# Multiple on same line
import tempfile, os
with tempfile.TemporaryDirectory() as d1, tempfile.TemporaryDirectory() as d2:
print(f"Temp dirs: {d1}, {d2}")
# Both exist here
# Both cleaned up here
# Nested with exception handling
@contextmanager
def transaction(conn):
"""Database transaction context manager"""
try:
yield conn
conn.commit()
print("Transaction committed")
except Exception as e:
conn.rollback()
print(f"Transaction rolled back: {e}")
raise9. Reentrant and Reusable Context Managers
import threading
from contextlib import contextmanager
# Non-reentrant (default threading.Lock)
lock = threading.Lock()
# with lock:
# with lock: # DEADLOCK! Lock is not reentrant
# pass
# Reentrant lock
rlock = threading.RLock()
with rlock:
with rlock: # OK! RLock is reentrant
print("Nested RLock works!")
# Reusable context manager
@contextmanager
def reusable_timer():
import time
start = time.time()
yield
print(f"Time: {time.time() - start:.4f}s")
# Can reuse as many times as needed
timer_cm = reusable_timer() # New context manager instance needed each time
for i in range(3):
with reusable_timer():
time.sleep(0.01 * (i + 1))10. Async Context Managers
import asyncio
class AsyncDatabaseConnection:
"""Async context manager for database"""
async def __aenter__(self):
print("Opening async database connection")
await asyncio.sleep(0.1) # Simulate connection setup
self.connection = "async_db_conn"
return self.connection
async def __aexit__(self, exc_type, exc_val, exc_tb):
print("Closing async database connection")
await asyncio.sleep(0.05) # Simulate cleanup
if exc_type:
print(f"Rolling back due to: {exc_val}")
return False
async def main():
async with AsyncDatabaseConnection() as conn:
print(f"Using connection: {conn}")
await asyncio.sleep(0.1) # Simulate work
print("Work done")
asyncio.run(main())
# Output:
# Opening async database connection
# Using connection: async_db_conn
# Work done
# Closing async database connection
# Async contextmanager decorator
from contextlib import asynccontextmanager
@asynccontextmanager
async def async_timer(label=""):
import time
start = time.perf_counter()
try:
yield
finally:
elapsed = time.perf_counter() - start
print(f"{label}: {elapsed:.4f}s")
async def main2():
async with async_timer("Async operation"):
await asyncio.sleep(0.1)
asyncio.run(main2())
# Output: Async operation: 0.1003s11. Real-World: Temporary Environment
📝 Hindi Explanation
Temporary state context managers are very useful in testing. You temporarily change something, run the test, then restore everything — whether the test passes or fails.
12. Best Practices
Summary
| Tool | Use Case |
|---|---|
with statement | Resource management |
__enter__ / __exit__ | Class-based context manager |
@contextmanager | Generator-based context manager |
contextlib.suppress | Ignore specific exceptions |
contextlib.ExitStack | Dynamic/multiple context managers |
asynccontextmanager | Async context manager |
redirect_stdout | Capture print output |
Golden Rule: If you write try/finally for cleanup, consider using a context manager instead. Your code will be cleaner and safer!⚠️ Common Mistakes
❌ Mistake 1: Returning True in __exit__ without understanding
class BadSuppress:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
return True # ❌ ALL exceptions silently suppress ho jaayengi!
with BadSuppress():
raise RuntimeError("Critical error!") # Silently eaten! 😱
print("No error seen — but data might be corrupted!")No error seen — but data might be corrupted!
Fix: Only suppress specific exceptions that you can handle. Blindly returning True creates a debugging nightmare. 🐛❌ Mistake 2: yield bhoolna contextmanager decorator mein
from contextlib import contextmanager
@contextmanager
def broken_manager():
print("Setup done")
# ❌ yield bhool gaye — RuntimeError milega!
print("Cleanup done")
# with broken_manager(): # RuntimeError: generator didn't yield
# passRuntimeError: generator didn't yield
Fix: The function decorated with@contextmanagermust have exactly oneyield.
❌ Mistake 3: Reusing a Context Manager when it's single-use
from contextlib import contextmanager
@contextmanager
def single_use_resource():
print("Acquired")
yield "resource"
print("Released")
cm = single_use_resource()
with cm:
print("First use — OK")
# with cm: # ❌ RuntimeError: generator already executing or exhausted
# print("Second use — FAIL!")Acquired First use — OK Released
Fix: Create a new context manager instance each time, or use a class-based approach that is reentrant.
❌ Mistake 4: Forgetting to return self from __enter__
class ForgetfulManager:
def __enter__(self):
print("Entering...")
# ❌ return bhool gaye — `as` variable None hoga!
def __exit__(self, *args):
return False
with ForgetfulManager() as obj:
print(f"obj is: {obj}") # None! 😱Entering... obj is: None
Fix: Always return a meaningful value from__enter__— usuallyselfor the managed resource.
❌ Mistake 5: Writing cleanup code after yield without try/finally
from contextlib import contextmanager
@contextmanager
def unsafe_manager():
resource = "acquired"
print(f"Resource: {resource}")
yield resource
# ❌ If an exception occurs in the with block, this code will be SKIPPED!
print("Cleanup done") # May never execute!
# Safe version:
@contextmanager
def safe_manager():
resource = "acquired"
print(f"Resource: {resource}")
try:
yield resource
finally:
print("Cleanup done") # ✅ ALWAYS executesFix: Always place cleanup code inside a try/finally block so that resources are released even in case of an exception.❌ Mistake 6: Getting the order wrong in nested context managers
# ❌ Wrong order — inner resource acquire before outer is ready
# with DatabaseTransaction() as txn, QueryBuilder(txn) as qb:
# ... # If txn setup fails, qb will also fail
# ✅ Correct — ensure dependencies are in right order
# with DatabaseConnection() as conn:
# with DatabaseTransaction(conn) as txn:
# with QueryBuilder(txn) as qb:
# qb.execute("SELECT * FROM users")Fix: Keep the order of dependent resources correct — outer resource first, inner resources after.
❌ Mistake 7: Raising exception in __exit__
class DangerousExit:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
raise RuntimeError("Cleanup failed!") # ❌ Original exception lost!
return False
# If a ValueError occurs in the with block, it will be lost
# and only RuntimeError will be shown — debugging impossible!Fix: Wrap cleanup code in __exit__ with try/except. Don't mask the original exception.✅ Key Takeaways
- 🔑 Context managers with the
withstatement guarantee resource management — file close, connection release, lock release all happen automatically.
- 🔑
__enter__method performs setup and returns a value that is assigned to theasvariable.__exit__performs cleanup whether or not an exception occurs.
- 🔑
@contextmanagerdecorator is best for simple cases — setup before yield, cleanup after yield. For complex state management, use the class-based approach.
- 🔑 Returning
Truefrom__exit__suppresses the exception. Only do this when you are consciously handling a specific exception.
- 🔑
contextlib.suppress()is for cleanly ignoring specific expected exceptions — more readable than try/except/pass.
- 🔑
ExitStackmanages a dynamic number of resources — for when you don't know at compile time how many context managers you'll need.
- 🔑 Async context managers (
async with,__aenter__/__aexit__) manage resources in async code — database connections, HTTP sessions, etc.
- 🔑 If you see a
try/finallypattern, consider creating a context manager — the code will be cleaner, reusable, and testable.
- 🔑 Always place cleanup code in
try/finallywhen using@contextmanager— otherwise cleanup will be skipped on exception.
- 🔑 Resources close in LIFO (Last In, First Out) order in nested context managers — this is safe for dependency chains.
❓ FAQ
Q1: What is the difference between a Context Manager and try/finally?
Answer: They perform functionally the same job, but a context manager is reusable, composable, and readable. try/finally is inline — you have to copy-paste it everywhere. Context managers are the Pythonic approach.
Q2: Can you use one context manager inside another?
Answer: Absolutely! Nested with statements are perfectly valid. In Python 3.1+, there's also comma-separated syntax: with open('a') as f1, open('b') as f2:. For dynamic nesting, use ExitStack.
Q3: When is it safe to return True in __exit__?
Answer: Only when you are handling specific, expected exceptions (like FileNotFoundError or StopIteration). Never return generic True — it hides bugs and makes debugging a nightmare.
Q4: Can you use multiple yield in @contextmanager?
Answer: No! There must be exactly one yield. Multiple yields will cause a RuntimeError. If you need multiple values, yield a tuple/object: yield (resource1, resource2).
Q5: Is the as clause optional in a context manager?
Answer: Yes! with open("file.txt"): is valid — if you don't need the returned value, you can skip as. __enter__ will still be called, but the return value will be discarded.
Q6: Are context managers thread-safe?
Answer: Not by default. If multiple threads share the same context manager instance, race conditions can occur. To make it thread-safe, use threading.Lock() or threading.RLock() inside the context manager.
Q7: contextlib.closing() kab use karein?
Answer: When an object has a close() method but doesn't implement the context manager protocol. with closing(urllib.request.urlopen(url)) as page: — this ensures close() is called automatically on exit.
Q8: What is the difference between ExitStack and nested with?
Answer: Use nested with when resources are fixed at compile time. Use ExitStack when the number of resources is determined at runtime — like a list of files or conditional resource acquisition.
🎯 Interview Questions
Q1: What is the Context Manager Protocol? Explain __enter__ and __exit__ methods.
Answer: The Context Manager Protocol is Python's resource management pattern. __enter__() acquires the resource and returns a value (which is assigned to the as variable), while __exit__() releases the resource (file close, lock release, connection close). Python guarantees __exit__ is always called.
Q2: How does the @contextmanager decorator work internally?
Answer: @contextmanager wraps a generator function into a context manager class. Code before yield becomes __enter__, the yielded value becomes the as variable, and code after yield becomes __exit__. If an exception occurs, it's thrown into the generator at the yield point.
Q3: Explain the parameters of __exit__. When should you return True vs False?
Answer: __exit__(self, exc_type, exc_val, exc_tb) — if no exception occurred, all three are None. exc_type is the exception class, exc_val is the exception instance, exc_tb is the traceback. Return True only when you intentionally handle a specific exception. Return False (or None) to let the exception propagate.
Q4: What is ExitStack and what is its use case?
Answer: ExitStack is a context manager that dynamically manages multiple context managers. Use cases: (1) Opening a variable number of files at runtime, (2) Conditional resource acquisition, (3) Managing cleanup callbacks.
Q5: How do you create an Async Context Manager? What's different from regular ones?
Answer: An async context manager implements __aenter__ and __aexit__ methods (which are coroutines — async def). It's used with the async with statement. The difference from regular context managers is that setup and teardown can themselves be awaited (e.g., async DB connection).
Q6: What's the similarity between a context manager and a decorator? Can a single class implement both?
Answer: Both implement a setup/teardown pattern. Yes, a single class can be both:
class Timed:
def __init__(self, label):
self.label = label
def __enter__(self):
self.start = time.time()
return self
def __exit__(self, *args):
print(f"{self.label}: {time.time() - self.start:.4f}s")
return False
def __call__(self, func):
@wraps(func)
def wrapper(*args, **kwargs):
with self:
return func(*args, **kwargs)
return wrapper
# As context manager: with Timed("op"): ...
# As decorator: @Timed("op")Q7: How does contextlib.suppress work internally?
Answer: suppress(*exceptions) is a context manager whose __exit__ method checks whether the raised exception is among the specified exceptions. If yes, it returns True (suppressing it). If no, it returns False (letting it propagate).
Q8: What are the risks of not using a context manager for file handling?
Answer: (1) Resource leak — file descriptors can be exhausted (OS limit), (2) Data loss — buffered data won't be flushed if the file isn't closed, (3) Permission issues — other processes may not be able to access the file, (4) Memory leak — large file objects stay in memory until garbage collected.
Q9: What happens if the inner __exit__ raises an exception in nested context managers?
Answer: If the inner context manager's __exit__ raises an exception, the outer context managers' __exit__ methods are still called (Python guarantees this). But the original exception may be masked — the new exception replaces it unless you handle it carefully.
Q10: How do you choose the right context manager design pattern for production code?
Answer: Decision tree: (1) Simple setup/teardown, no state → @contextmanager with try/finally, (2) Complex state, multiple methods → Class-based with __enter__/__exit__, (3) Dynamic resources → ExitStack, (4) Async code → @asynccontextmanager or async class.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Python Context Managers.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, advanced, context, managers
Related Python Master Course Topics