urllib3
urllib3 is the low-level HTTP client that most of the Python ecosystem is built on. It adds what the standard library's http.client lacks: thread-safe connection pooling, client-side TLS verification, multipart file uploads, configurable retries, redirect handling, and gzip/brotli/zstd decoding. requests, botocore, and much of PyPI's most-installed software route their traffic through it. Version 2 added a top-level urllib3.request() helper and a response .json() method, so simple calls no longer require constructing a PoolManager yourself. It is synchronous only, HTTP/1.1 only, and deliberately lower-level than the clients built on top of it.
The plumbing under most Python HTTP traffic, maintained with unusual care, and the right direct dependency for libraries and thread-heavy code that wants control. For everyday application code, requests or httpx on top of it will cost you nothing and read better.
Use it if
- You are writing a library or SDK and want an HTTP dependency with a 15+ year security track record and no higher-level baggage like cookies-by-default or auth helpers
- You need precise control over connection pools: pool size per host, blocking behavior when the pool is exhausted, socket options, or separate connect and read timeouts
- You need per-request retry policy with backoff and status allowlists via urllib3.Retry, which is more configurable than what requests exposes directly
- Your code runs in many threads and you want one shared PoolManager reusing sockets safely across all of them
- You are writing application code: requests or httpx sit on the same foundations with far nicer ergonomics for sessions, auth, and cookies, and the extra dependency is already on most machines
- You need async: urllib3 is blocking, full stop. httpx or aiohttp are the right tools for asyncio code
- You need HTTP/2 or HTTP/3: urllib3 speaks HTTP/1.1 only; httpx has HTTP/2 support today
- You maintain code written against urllib3 1.x: v2 removed and renamed parameters (method allowlists, strict mode) and requires OpenSSL 1.1.1+, and a large share of tutorials and Stack Overflow answers still show 1.x idioms that no longer work
Setup reality
pip install urllib3 is a pure-Python wheel with zero required dependencies, so installation itself never fails. The friction points: current 2.x releases require Python 3.10+, v2 requires OpenSSL 1.1.1 or newer and will refuse to import against ancient or exotic TLS builds (an issue on some old distros and embedded systems), and SOCKS proxies, brotli, and zstd each need an extra (pip install "urllib3[socks]"). Certificate verification is on by default in v2, which is correct but surprises people upgrading code that silently talked to self-signed endpoints. Responses default to preloading the whole body into memory; streaming requires preload_content=False plus remembering to release the connection.
Patterns
One-off GET with the top-level APIsimple-get
import urllib3
resp = urllib3.request("GET", "https://httpbin.org/robots.txt")
print(resp.status) # 200
print(resp.data) # raw bytesurllib3.request() is v2-only and uses a module-level shared PoolManager, so repeated calls to the same host reuse connections.
Send and parse JSONjson-request-response
import urllib3
resp = urllib3.request(
"POST",
"https://httpbin.org/post",
json={"name": "widget", "qty": 3},
)
data = resp.json()json= sets the Content-Type header for you and is mutually exclusive with body= and fields=. resp.json() exists in v2 only.
GET with query string parametersquery-parameters
import urllib3
http = urllib3.PoolManager()
resp = http.request(
"GET",
"https://httpbin.org/get",
fields={"q": "widgets", "page": "2"},
)For GET/HEAD/DELETE, fields= is URL-encoded into the query string. For POST it becomes multipart form data instead, which trips people up.
Shared connection pool for many requestspool-manager
import urllib3
http = urllib3.PoolManager(num_pools=10, maxsize=10, block=True)
for path in ("/a", "/b", "/c"):
resp = http.request("GET", f"https://api.example.com{path}")maxsize is connections kept per host, and only matters under threading. block=True makes threads wait for a free connection instead of creating throwaway ones.
Separate connect and read timeoutstimeouts
import urllib3
timeout = urllib3.Timeout(connect=2.0, read=5.0)
http = urllib3.PoolManager(timeout=timeout)
resp = http.request("GET", "https://api.example.com/slow")Without an explicit timeout, requests can hang for the OS socket default, which may be minutes. Set one on the PoolManager so every request inherits it.
Retry with exponential backoff on 5xxretries-with-backoff
import urllib3
from urllib3.util import Retry
retries = Retry(
total=3,
backoff_factor=0.5,
status_forcelist=[502, 503, 504],
)
http = urllib3.PoolManager(retries=retries)
resp = http.request("GET", "https://api.example.com/flaky")By default only idempotent methods are retried. Add allowed_methods={"GET", "POST"} if you really want POST retries, and know the duplicate-request risk.
Stream a large body without loading it in memorystream-large-download
import urllib3
http = urllib3.PoolManager()
resp = http.request("GET", "https://example.com/big.bin", preload_content=False)
with open("big.bin", "wb") as f:
for chunk in resp.stream(1024 * 64):
f.write(chunk)
resp.release_conn()Forgetting release_conn() (or resp.release_conn via a context manager) leaks the connection from the pool until garbage collection.
Upload a file as multipart form dataupload-file-multipart
import urllib3
with open("report.pdf", "rb") as f:
file_data = f.read()
resp = urllib3.request(
"POST",
"https://httpbin.org/post",
fields={"file": ("report.pdf", file_data, "application/pdf")},
)The tuple is (filename, data, content_type). urllib3 reads the whole file into memory first; for multi-GB uploads pass a body with a generator instead.
Verify TLS against a private CAcustom-ca-bundle
import urllib3
http = urllib3.PoolManager(ca_certs="/etc/ssl/internal-ca.pem")
resp = http.request("GET", "https://internal.example.com/health")Verification is on by default in v2. Use cert_reqs="CERT_NONE" only for throwaway debugging; it also emits an InsecureRequestWarning on every call.
Route requests through a proxyhttp-proxy
import urllib3
proxy = urllib3.ProxyManager(
"http://proxy.example.com:8080",
proxy_headers=urllib3.make_headers(proxy_basic_auth="user:pass"),
)
resp = proxy.request("GET", "https://httpbin.org/ip")ProxyManager replaces PoolManager entirely. SOCKS proxies need pip install "urllib3[socks]" and SOCKSProxyManager from urllib3.contrib.socks.
Set headers for every request in a pooldefault-headers
import urllib3
http = urllib3.PoolManager(
headers={"User-Agent": "my-tool/1.0", "Authorization": "Bearer TOKEN"}
)
resp = http.request("GET", "https://api.example.com/me")Per-request headers= replaces the pool headers for that call rather than merging with them, so re-send the Authorization header if you override.