requests
requests is the standard synchronous HTTP client for Python. It wraps urllib3 in an API that reads like the HTTP you mean: requests.get(url, auth=(user, pass)) returns a Response with .status_code, .headers, and .json(). Sessions add connection pooling, cookie persistence, and shared defaults. It supports Python 3.10+ and, per its own README, is depended on by over 4 million repositories, which makes it close to a de facto part of the language.
Still the right default for synchronous HTTP in Python: boring, everywhere, and documented to death. Reach for httpx the moment async or HTTP/2 enters the picture.
Use it if
- You need to call HTTP APIs from scripts, cron jobs, or synchronous backends with zero learning curve
- You want Sessions with connection pooling, cookie persistence, and mountable retry behavior from urllib3
- You need streaming downloads, multipart uploads, digest auth, or SOCKS proxies without hunting for extra libraries
- You are writing example code or docs that other people must be able to run anywhere
- Your code is async. requests is blocking only; httpx gives you a nearly identical API with async support, and aiohttp is the asyncio native option
- You need HTTP/2 or HTTP/3; requests speaks HTTP/1.1 only
- You assume retries and timeouts are built in. There is no default timeout (a classic production incident) and retries require manually mounting an HTTPAdapter
- You want a library that is still growing. requests is deliberately conservative; fixes land, new features mostly do not
Setup reality
pip install requests pulls in urllib3, certifi, charset_normalizer, and idna and works on any Python 3.10+ without native builds. The friction is policy, not packaging: no timeout is applied unless you pass one on every single call, retries mean wiring an HTTPAdapter with a urllib3 Retry object yourself, and .json() raises on non-JSON bodies so error paths need care. None of it is hard; all of it is on you.
Patterns
GET a URL and parse JSONget-json
import requests
r = requests.get("https://api.example.com/items", timeout=10)
r.raise_for_status()
data = r.json()r.json() raises if the body is not JSON; check r.status_code or call raise_for_status() first so you fail on the real error.
POST a JSON bodypost-json
import requests
r = requests.post(
"https://api.example.com/items",
json={"name": "widget", "qty": 3},
timeout=10,
)Use json=, not data=; json= serializes and sets the Content-Type header for you, data= sends form encoding.
Always set a timeoutset-timeout
import requests
# (connect timeout, read timeout)
r = requests.get("https://api.example.com/slow", timeout=(3.05, 27))There is no default timeout; a request without one can hang a worker forever. This is the most common requests bug in production.
Reuse a Session for connection poolingreuse-session
import requests
s = requests.Session()
s.headers.update({"Authorization": "Bearer TOKEN"})
for page in range(1, 6):
r = s.get("https://api.example.com/items", params={"page": page}, timeout=10)
r.raise_for_status()A Session keeps TCP connections alive across calls; calling requests.get in a loop reopens a connection every time.
Retry failed requests with backoffretry-with-backoff
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
s = requests.Session()
retry = Retry(
total=3,
backoff_factor=0.5,
status_forcelist=[429, 500, 502, 503, 504],
)
s.mount("https://", HTTPAdapter(max_retries=retry))
r = s.get("https://api.example.com/data", timeout=10)Retries are not built into requests itself; you mount urllib3's Retry. By default only idempotent methods are retried.
Stream a large file to diskstream-download
import requests
with requests.get(url, stream=True, timeout=30) as r:
r.raise_for_status()
with open("file.zip", "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)Without stream=True the entire body is read into memory before you see it; with it, always use a with block so the connection is released.
Upload a file as multipart form dataupload-file
import requests
with open("report.pdf", "rb") as f:
r = requests.post(
"https://api.example.com/upload",
files={"file": ("report.pdf", f, "application/pdf")},
timeout=60,
)Open the file in binary mode; passing files= sets the multipart boundary and Content-Type automatically.
Handle HTTP and network errors separatelyhandle-errors
import requests
try:
r = requests.get("https://api.example.com/items", timeout=10)
r.raise_for_status()
except requests.exceptions.HTTPError as e:
print("bad status:", e.response.status_code)
except requests.exceptions.ConnectionError:
print("network problem")
except requests.exceptions.Timeout:
print("timed out")A 404 does not raise by itself; requests only raises for network-level failures unless you call raise_for_status().
Send query string parametersquery-params
import requests
r = requests.get(
"https://api.example.com/search",
params={"q": "blue widgets", "page": 2},
timeout=10,
)
print(r.url) # https://api.example.com/search?q=blue+widgets&page=2Use params= instead of building the query string by hand; it handles encoding and lists correctly.
Authenticate with basic authbasic-auth
import requests
r = requests.get(
"https://httpbin.org/basic-auth/user/pass",
auth=("user", "pass"),
timeout=10,
)
print(r.status_code) # 200auth=(user, pass) is basic auth; digest auth needs requests.auth.HTTPDigestAuth instead.
Route requests through a proxyuse-proxies
import requests
proxies = {
"http": "http://10.10.1.10:3128",
"https": "http://10.10.1.10:1080",
}
r = requests.get("https://api.example.com", proxies=proxies, timeout=10)requests also honors HTTP_PROXY/HTTPS_PROXY environment variables, which can surprise you in CI; SOCKS needs the socks extra installed.