selenium
Selenium is the original browser automation project and the reference implementation of the W3C WebDriver specification: a language-neutral protocol for driving real Chrome, Firefox, Edge and Safari. The selenium PyPI package is the official Python binding. You write find-element and interaction code, the driver speaks WebDriver to an actual browser, and since Selenium Manager arrived the matching browser driver is downloaded for you automatically.
Still the standard where real cross-browser coverage, Grid infrastructure or non-mainstream language bindings matter, and 4.x is well maintained. For a brand new Python test suite, Playwright is the more productive default; choose Selenium deliberately, not by inertia.
Use it if
- You must test on the real browsers users run, including Safari and IE-mode Edge, through the one standardized protocol every vendor implements
- You run tests at scale on Selenium Grid or commercial clouds (BrowserStack, Sauce Labs) that are built around WebDriver
- You maintain an existing Selenium suite; the 4.x line is stable and there is fifteen-plus years of answers for every error you will hit
- You need automation in a language Playwright does not cover; Selenium bindings exist for Java, Python, C#, Ruby, JavaScript and Kotlin
- You are starting a new test suite from zero; Playwright gives you auto-waiting, network interception and tracing out of the box, while in Selenium reliable waiting is your job and flaky tests are the tax for skipping it
- You are scraping mostly static pages; requests plus an HTML parser is orders of magnitude faster than booting a real browser per page
- You want built-in test-runner features like retries, parallelism and fixtures; Selenium is only the browser driver and you assemble the rest from pytest and plugins
- You need to intercept or mock network traffic; WebDriver's answer (BiDi) is still maturing and far less ergonomic than devtools-native tools
Setup reality
pip install selenium and webdriver.Chrome() genuinely works now: Selenium Manager fetches the right chromedriver automatically, killing the version-mismatch misery of the 3.x era. The lasting pain is synchronization: nothing auto-waits, so every dynamic page needs explicit WebDriverWait code or you inherit intermittent NoSuchElement and StaleElementReference failures. CI needs headless flags plus window-size set, and browser updates can still shift timing enough to expose weak waits.
Patterns
Start and stop a browserstart-driver
from selenium import webdriver
driver = webdriver.Chrome() # Selenium Manager fetches the driver binary
driver.get('https://example.com')
print(driver.title)
driver.quit()No manual chromedriver download needed since Selenium Manager; always call quit() (not close()) or orphan browser processes pile up.
Headless Chrome for CIheadless-ci
from selenium import webdriver
opts = webdriver.ChromeOptions()
opts.add_argument('--headless=new')
opts.add_argument('--window-size=1920,1080')
opts.add_argument('--no-sandbox') # required in most Docker images
driver = webdriver.Chrome(options=opts)Headless defaults to a small viewport; unset window-size is a classic source of element-not-clickable failures that only happen in CI.
Locate elements with Byfind-elements
from selenium.webdriver.common.by import By
button = driver.find_element(By.CSS_SELECTOR, 'button.submit')
links = driver.find_elements(By.TAG_NAME, 'a')
field = driver.find_element(By.NAME, 'email')find_element raises NoSuchElementException immediately; find_elements returns an empty list, which is the cheap way to test presence.
Wait for an element properlyexplicit-wait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
el = wait.until(EC.element_to_be_clickable((By.ID, 'save')))
el.click()This is the single most important Selenium pattern; time.sleep calls are how suites become slow AND flaky at the same time.
Fill and submit a formfill-form
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
user = driver.find_element(By.NAME, 'username')
user.clear()
user.send_keys('alice')
driver.find_element(By.NAME, 'password').send_keys('secret' + Keys.RETURN)Call clear() before send_keys on prefilled inputs; send_keys appends rather than replaces.
Pick an option from a select elementselect-dropdown
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
select = Select(driver.find_element(By.ID, 'country'))
select.select_by_visible_text('Germany')Select only works on real <select> tags; custom JS dropdowns need normal click-and-wait interaction.
Run JavaScript in the pageexecute-script
height = driver.execute_script('return document.body.scrollHeight')
driver.execute_script('window.scrollTo(0, arguments[0])', height)
driver.execute_script('arguments[0].scrollIntoView({block: "center"})', element)Pass Python values via arguments[] instead of string formatting; it handles escaping and element references correctly.
Capture page and element screenshotsscreenshots
driver.save_screenshot('page.png')
el = driver.find_element(By.ID, 'chart')
el.screenshot('chart.png')Screenshots capture the viewport, not the full page; hook them into test-failure teardown for debuggable CI runs.
Switch into iframes and new tabsiframes-windows
from selenium.webdriver.common.by import By
frame = driver.find_element(By.CSS_SELECTOR, 'iframe#payment')
driver.switch_to.frame(frame)
# ... interact inside the iframe ...
driver.switch_to.default_content()
original = driver.current_window_handle
for handle in driver.window_handles:
if handle != original:
driver.switch_to.window(handle)Elements inside an iframe are invisible until you switch into it; forgetting switch_to.default_content() breaks every locator afterwards.
Survive StaleElementReferenceExceptionhandle-stale-elements
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.common.by import By
def click_fresh(driver, locator, attempts=3):
for _ in range(attempts):
try:
driver.find_element(*locator).click()
return
except StaleElementReferenceException:
continue
raise TimeoutError(f'element kept going stale: {locator}')
click_fresh(driver, (By.ID, 'refresh-prone-button'))Staleness means the DOM node was replaced after you found it; re-find the element instead of caching WebElement objects across page updates.
Reuse a login session via cookiescookies-session
import json
# after logging in once
json.dump(driver.get_cookies(), open('cookies.json', 'w'))
# in a later session
driver.get('https://example.com') # must be on the domain first
for c in json.load(open('cookies.json')):
driver.add_cookie(c)
driver.refresh()add_cookie only works after navigating to the cookie's domain; adding cookies on about:blank throws.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| playwright | PyPI | New projects: auto-waiting, tracing and network mocking with less flake for most teams |
| beautifulsoup4 | PyPI | Scraping server-rendered HTML where no JavaScript needs to execute |
| helium | PyPI | You want a friendlier high-level wrapper while keeping Selenium underneath |