Python Notes
Master Selenium WebDriver with Python for browser automation, form filling, clicking, screenshots, handling dynamic content, waits, and headless browsing with Hindi explanations.
Introduction
Selenium is a library for programmatically controlling browsers. Dynamic content loaded via JavaScript, login forms, button clicks — all of this can be automated with Selenium.
pip install selenium
pip install webdriver-manager # Auto ChromeDriver download2. Finding Elements
Total links: 18 Text: More information..., href: https://www.iana.org/domains/example Text: , href: https://www.google.com/intl/en/about.html Text: Store, href: https://store.google.com
3. Interacting with Elements
Size options: ['Small', 'Medium', 'Large'] Current URL: https://httpbin.org/post
4. Waits — Handling Dynamic Content
Found h1: Herman Melville - Moby Dick
📝 Hindi Explanation
Wait is THE MOST important concept in Selenium. Modern websites load content dynamically via JavaScript.implicitly_wait(n)globally waits for all elements, whileWebDriverWait(explicit wait) waits for specific conditions — this is the recommended approach.
5. Screenshots and Page Source
Screenshot saved!
Page text: Example Domain
This domain is for use in illustrative examples in documents...
Page height: 600px
Element position: {'x': 32, 'y': 100.5, 'width': 600, 'height': 38}6. Handling Alerts, Frames, Windows
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)
wait = WebDriverWait(driver, 10)
# Alert handling
def handle_alert(driver, action="accept"):
try:
alert = wait.until(EC.alert_is_present())
print(f"Alert text: {alert.text}")
if action == "accept":
alert.accept() # OK button
elif action == "dismiss":
alert.dismiss() # Cancel button
elif action == "input":
alert.send_keys("input text") # Prompt mein text
alert.accept()
except Exception as e:
print(f"No alert: {e}")
# Frame/iframe handling
def switch_to_frame_and_back(driver, frame_selector):
# Switch to iFrame
frame = driver.find_element(By.CSS_SELECTOR, frame_selector)
driver.switch_to.frame(frame)
# Work inside the iFrame
content = driver.find_element(By.TAG_NAME, "body").text
# Main document pe wapas
driver.switch_to.default_content()
return content
# Multiple windows/tabs
def open_and_switch_window(driver, url):
# Naya tab kholo
original_window = driver.current_window_handle
driver.execute_script(f"window.open('{url}', '_blank')")
# New window pe switch karo
for window in driver.window_handles:
if window != original_window:
driver.switch_to.window(window)
break
print(f"New window title: {driver.title}")
# Original window pe wapas
driver.switch_to.window(original_window)
driver.quit()7. Actions Chains — Complex Interactions
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)
driver.get("https://example.com")
actions = ActionChains(driver)
# Hover
element = driver.find_element(By.TAG_NAME, "h1")
actions.move_to_element(element).perform()
# Click at offset
actions.move_to_element_with_offset(element, 50, 10).click().perform()
# Drag and drop (simulated)
source = driver.find_element(By.ID, "source") # If element exists
target = driver.find_element(By.ID, "target")
# actions.drag_and_drop(source, target).perform()
# Complex action chain
# actions.move_to_element(menu)
# .click()
# .move_to_element(submenu)
# .click()
# .perform()
# Keyboard actions
from selenium.webdriver.common.keys import Keys
body = driver.find_element(By.TAG_NAME, "body")
actions.key_down(Keys.CONTROL).send_keys("a").key_up(Keys.CONTROL).perform() # Ctrl+A
driver.quit()8. Headless Browser Automation
Title: Example Domain Links found: 1
9. Page Object Model (POM)
Summary
| Feature | Method |
|---|---|
| Navigate | driver.get(url) |
| Find element | find_element(By.X, "selector") |
| Find multiple | find_elements(By.X, "selector") |
| Click | element.click() |
| Type text | element.send_keys("text") |
| Get text | element.text |
| Get attribute | element.get_attribute("href") |
| Screenshot | driver.save_screenshot("x.png") |
| Wait | WebDriverWait(driver, 10).until(EC...) |
| Run JS | driver.execute_script("...") |
| Quit | driver.quit() |
When to use Selenium vs requests? - requests + BS4: Static HTML, APIs — faster, simpler - Selenium: JavaScript-rendered content, login forms, clicking buttons, testing UIs
⚠️ Common Mistakes
❌ Mistake 1: time.sleep() ka overuse
# ❌ WRONG — fixed sleep unreliable hai
import time
driver.get("https://example.com")
time.sleep(5) # Page load ho ya na ho, 5 sec wait karega
element = driver.find_element(By.ID, "content")
# ✅ RIGHT — Explicit wait use karo
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "content"))
)💡 time.sleep() makes tests slow and flaky. WebDriverWait dynamically waits until the condition becomes true.❌ Mistake 2: Not calling driver.quit()
# ❌ WRONG — browser process leak hoga
driver = webdriver.Chrome()
driver.get("https://example.com")
# ... script ends without quit
# ✅ RIGHT — HAMESHA try/finally ya context manager use karo
try:
driver = webdriver.Chrome()
driver.get("https://example.com")
finally:
driver.quit()💡 If you don't call driver.quit(), Chrome processes will accumulate and slow down the system.❌ Mistake 3: Forgetting to switch frames
# ❌ WRONG — element inside iframe won't be found directly
element = driver.find_element(By.ID, "iframe-content") # NoSuchElementException!
# ✅ RIGHT — switch to frame first
driver.switch_to.frame("my-iframe")
element = driver.find_element(By.ID, "iframe-content")
driver.switch_to.default_content() # Wapas main page pe❌ Mistake 4: Not handling Stale Element Reference
# ❌ WRONG — after page refresh, old element reference becomes invalid
element = driver.find_element(By.ID, "btn")
driver.refresh()
element.click() # StaleElementReferenceException!
# ✅ RIGHT — re-find karo element
driver.refresh()
element = driver.find_element(By.ID, "btn")
element.click()❌ Mistake 5: Mixing implicit and explicit waits
# ❌ WRONG — mixing both causes unpredictable behavior
driver.implicitly_wait(10)
wait = WebDriverWait(driver, 5) # Conflict!
# ✅ RIGHT — ek approach choose karo (Explicit recommended)
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID, "btn")))❌ Mistake 6: Not setting window size in headless mode
# ❌ WRONG — default window size is small in headless mode
options.add_argument("--headless=new")
# Elements visible nahi honge, responsive layout break hoga
# ✅ RIGHT — explicit window size set karo
options.add_argument("--headless=new")
options.add_argument("--window-size=1920,1080")❌ Mistake 7: Hard-coded XPaths use karna
💡 Never use absolute XPath. Prefer IDs, data attributes, or meaningful CSS selectors.
✅ Key Takeaways
- 🔑 WebDriver = Real Browser Control — Selenium programmatically operates an actual Chrome/Firefox browser, unlike the requests library
- 🔑 Headless Mode is essential for production scraping — the
--headless=newflag runs the browser without a GUI - 🔑 Explicit Waits > Implicit Waits > time.sleep() —
WebDriverWait+ Expected Conditions is the most reliable approach - 🔑 Locator Strategy Priority: ID > CSS Selector > XPath > Class Name > Tag Name — specificity matters
- 🔑 Page Object Model (POM) is a MUST for large test suites — improves maintainability and reusability
- 🔑 ALWAYS call
driver.quit()— to avoid memory leaks and zombie processes - 🔑 Anti-detection measures are necessary for scraping — hide WebDriver property, set realistic user-agent
- 🔑 ActionChains are for complex interactions (hover, drag-drop, right-click) — more powerful than simple
.click() - 🔑 Understanding frame/window switching is essential — nested content isn't accessible without
switch_to.frame()andswitch_to.window() - 🔑 Screenshots are lifesavers during debugging — use
save_screenshot()to capture visual state when tests fail
❓ FAQ
Q1: Selenium vs Playwright vs Puppeteer — kaunsa choose karein?
A: Selenium is the oldest and most widely supported (Java, Python, C#, JS). Playwright is modern, faster, and has better auto-wait. Puppeteer is Chrome-only but very fast for Node.js. Choose based on team skills and requirements.
Q2: How to fix ChromeDriver version mismatch error?
A: Use the webdriver-manager library — it automatically downloads the correct version:
from webdriver_manager.chrome import ChromeDriverManager
service = Service(ChromeDriverManager().install())If managing manually, the Chrome browser version and ChromeDriver version must match.
Q3: Website is detecting Selenium — how to bypass?
A: Multiple techniques combine karo:
navigator.webdriverproperty hide karo via CDP command- Realistic User-Agent set karo
--disable-blink-features=AutomationControlledflag add karo- Random delays add karo between actions
- Note: Scrape ethically — respect the website's ToS!
Q4: How to scrape infinite scroll pages?
A: Execute JavaScript to scroll and wait for new content to load:
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
time.sleep(2)
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_heightQ5: How to upload files in Selenium?
A: input[type='file'] element pe directly send_keys() use karo:
There's no need to make hidden inputs visible — send_keys directly sends the path.
Q6: How to handle multiple tabs/windows?
A: driver.window_handles gives a list of all open windows. switch_to.window(handle) switches to a specific window. For a new tab, use driver.execute_script('window.open()') and switch to the new handle.
Q7: How to access Shadow DOM elements?
A: Selenium 4+ mein shadow_root property available hai:
shadow_host = driver.find_element(By.CSS_SELECTOR, "#shadow-host")
shadow_root = shadow_host.shadow_root
inner_element = shadow_root.find_element(By.CSS_SELECTOR, "#inner-content")Q8: How to integrate Selenium tests into a CI/CD pipeline?
A: Enable headless mode, use a Docker container (selenium/standalone-chrome), and run with pytest. In GitHub Actions / Jenkins, use xvfb (virtual display) for non-headless execution if needed.
🎯 Interview Questions
Q1: What is Selenium WebDriver and how is it different from Selenium RC?
A: Selenium WebDriver directly calls the browser's native APIs via browser-specific drivers (ChromeDriver, GeckoDriver). Selenium RC (deprecated) injected JavaScript into the browser — slower, less reliable. WebDriver is the modern, W3C standardized approach.
Q2: What is the difference between Implicit Wait, Explicit Wait, and Fluent Wait?
A:
- Implicit Wait: Set globally. Polls for the specified time on every
find_elementcall.driver.implicitly_wait(10)— searches for element up to 10 sec. - Explicit Wait: Waits for a specific condition.
WebDriverWait(driver, 10).until(EC.element_to_be_clickable(...))— gives precise control. - Fluent Wait: Advanced version of Explicit Wait — offers custom polling interval and the ability to ignore specific exceptions. Explicit Wait is recommended for production.
Q3: What is Page Object Model (POM) and what are its advantages?
A: POM is a design pattern where each web page is represented by a class. The page's elements and actions are encapsulated within that class. Advantages:
- Maintainability — when UI changes, only one place needs updating
- Reusability — the same page object is used across multiple tests
- Readability — test code stays clean and business-logic focused
- Separation of Concerns — locators and test logic are kept separate
Q4: What is Stale Element Reference Exception and how to handle it?
A: This exception occurs when the DOM is refreshed/re-rendered and the old element reference becomes invalid. Solution: Re-find the element after DOM changes, or use retry logic with explicit waits.
from selenium.common.exceptions import StaleElementReferenceException
def click_with_retry(driver, locator, retries=3):
for i in range(retries):
try:
driver.find_element(*locator).click()
return
except StaleElementReferenceException:
if i == retries - 1:
raiseQ5: What are the different types of locators in Selenium? What should the priority order be?
A: Locators: By.ID, By.NAME, By.CLASS_NAME, By.TAG_NAME, By.CSS_SELECTOR, By.XPATH, By.LINK_TEXT, By.PARTIAL_LINK_TEXT. Priority: ID > Name > CSS Selector > XPath > Others. ID is fastest because it's unique in the DOM. CSS Selector parses faster than XPath. XPath is most flexible but slowest and most fragile.
Q6: What are the advantages and disadvantages of headless browser testing?
A: Advantages: Faster execution, CI/CD friendly, no GUI dependency, less resource usage. Disadvantages: Visual bugs aren't caught, some sites detect headless mode, debugging is harder (need to take screenshots), and rendering differences may exist compared to headed mode.
Q7: How do you handle dynamic web elements in Selenium?
A: For dynamic elements:
- Explicit Waits — wait for element to appear
- Dynamic XPath/CSS —
contains(),starts-with()use karo - JavaScript Executor — access directly via
execute_script() - Retry logic — retry mechanism for StaleElement
- Fluent Wait — custom polling frequency set karo
Q8: What is Selenium Grid and when do you use it?
A: Selenium Grid is a distributed testing framework that runs parallel tests on multiple machines/browsers. The Hub is the central server that distributes tests. Nodes are machines with actual browsers. Use it when you need cross-browser testing, parallel execution for faster CI/CD, or testing on different OS versions.
Q9: What is the difference between driver.close() and driver.quit()?
A: driver.close() closes only the current window/tab. driver.quit() closes all windows AND ends the WebDriver session (terminates the browser process). Always use quit() in cleanup to prevent resource leaks.
Q10: When and how do you take screenshots in Selenium? How does it help with debugging?
A: Capture screenshots automatically at test failure time:
import pytest
@pytest.fixture(autouse=True)
def screenshot_on_failure(request, driver):
yield
if request.node.rep_call.failed:
driver.save_screenshot(f"screenshots/{request.node.name}.png")In debugging, screenshots are essential for seeing page state inside headless mode. Element-level screenshot (element.screenshot()) helps inspect specific UI components in isolation.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Selenium Web Automation with Python.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, automation, selenium, selenium web automation with python
Related Python Master Course Topics