backoff
backoff is a dependency-free Python module that gives you two decorators for retrying flaky work. @backoff.on_exception retries when the wrapped call raises one of the exception types you name; @backoff.on_predicate retries when the return value fails a test, which is how you poll for content that is not there yet. You choose a wait generator (expo, fibo, constant, decay, or runtime, which reads a delay out of the response), a give-up condition (max_tries, max_time, or a giveup callable that inspects the exception), and optional on_backoff, on_giveup, and on_success handlers for logging or metrics. The same decorators work on plain functions and on async coroutines.
A well-designed library that is finished in the bad sense: archived upstream, no release since 2022, and no path for a fix if you find a bug. Keep it where it already works, and reach for tenacity or stamina in anything new.
Use it if
- You are already using it and it works: the code is small, has zero dependencies, and 2.2.1 still installs and runs fine on current Python
- You need to poll for externally generated content, where @backoff.on_predicate retrying on a falsy return value expresses the job more directly than exception-based retry libraries
- You want one decorator that covers both sync functions and asyncio coroutines without a second API or a separate package
- You want AWS-style Full Jitter by default rather than having to configure a jitter strategy yourself
- You are starting something new: the GitHub repository was archived in May 2024 and is read-only, the last release (2.2.1) shipped in October 2022, and the 43 open issues can never receive a reply
- You want a maintained equivalent: tenacity and stamina are actively released, cover the same ground, and both give you a retry context manager so you can retry a block without extracting it into a function
- You need to retry inline code: backoff is decorator-only, so every retryable unit has to become its own function first
- Your resolver is strict about metadata: the package declares python = ">=3.7,<4.0" and its classifiers stop at Python 3.10, so it advertises support for nothing released in the last several years even though it runs
- You need per-attempt timeouts, retry budgets, or circuit breaking: none of that exists here, and bolting it onto a frozen library means vendoring it
Setup reality
pip install backoff is instant and pulls nothing: it is pure Python with no dependencies. The surprises are behavioral, not installational. Decorator keyword arguments are evaluated at import time, so anything from runtime config has to be passed as a callable instead of a value. The default jitter is full_jitter, which means the number the wait generator yields is the maximum wait, not the actual one, so max_tries with expo gives you a wide and unpredictable total duration unless you also set max_time. Nothing is logged until you attach a handler, because the 'backoff' logger ships with a NullHandler. And raise_on_giveup=False makes the decorated function quietly return None after the final failure, which is easy to mistake for success.
Patterns
Retry a call when it raisesretry-on-exception
import backoff
import requests
@backoff.on_exception(backoff.expo, requests.exceptions.RequestException)
def get_url(url):
return requests.get(url)
# a tuple works too
@backoff.on_exception(
backoff.expo,
(requests.exceptions.Timeout, requests.exceptions.ConnectionError),
)
def get_url_narrow(url):
return requests.get(url)With no max_tries or max_time this retries forever. Never deploy the bare form: a permanently broken endpoint turns into a process that never returns and never errors.
Put a ceiling on retryinglimit-attempts
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_time=60, # total seconds, checked before each sleep
)
def get_url(url):
return requests.get(url)
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_tries=8,
jitter=None, # exact expo waits: 1, 2, 4, 8, 16, 32, 64
)
def get_url_fixed(url):
return requests.get(url)max_time is the honest control. With the default full_jitter, max_tries=8 can take anywhere from seconds to two minutes, because each yielded value is only an upper bound on the actual sleep.
Do not retry errors that will never succeedgiveup-condition
def fatal_code(e):
return 400 <= e.response.status_code < 500
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_time=300,
giveup=fatal_code,
)
def get_url(url):
return requests.get(url)Returning True from giveup means stop and re-raise. Guard against a missing response attribute: a ConnectionError has no .response, so this exact predicate raises AttributeError inside the retry machinery.
Retry based on the return valuepoll-on-predicate
@backoff.on_predicate(backoff.fibo, lambda x: x == [], max_value=13)
def poll_for_messages(queue):
return queue.get()
# the default predicate is a falsey test, so this is equivalent for []
@backoff.on_predicate(backoff.fibo, max_value=13)
def poll_short(queue):
return queue.get()
# fixed one-second polling
@backoff.on_predicate(backoff.constant, jitter=None, interval=1)
def poll_steady(queue):
return queue.get()Extra keyword arguments are forwarded to the wait generator, which is why max_value and interval sit next to the backoff options. Leave jitter on for anything hitting a shared service or every client polls in lockstep.
Wait as long as the server tells you torespect-retry-after
@backoff.on_predicate(
backoff.runtime,
predicate=lambda r: r.status_code == 429,
value=lambda r: int(r.headers.get("Retry-After")),
jitter=None,
)
def get_url():
return requests.get(url)The runtime generator receives whatever the call produced and returns your delay. int() on a missing header raises TypeError, and Retry-After is also allowed to be an HTTP date, so parse defensively in real code.
Retry coroutinesasync-retry
import aiohttp
import backoff
@backoff.on_exception(backoff.expo, aiohttp.ClientError, max_time=60)
async def get_url(url):
async with aiohttp.ClientSession(raise_for_status=True) as session:
async with session.get(url) as response:
return await response.text()The same decorators detect a coroutine and sleep with asyncio.sleep instead of time.sleep. Event handlers may be coroutines too, but mixing a sync handler into an async retry blocks the event loop for its duration.
Report every retry and give-upevent-handlers
def backoff_hdlr(details):
print("Backing off {wait:0.1f}s after {tries} tries "
"calling {target} with args {args}".format(**details))
def giveup_hdlr(details):
metrics.increment("retry.giveup", tags=[f"target:{details['target'].__name__}"])
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_tries=5,
on_backoff=backoff_hdlr,
on_giveup=giveup_hdlr,
)
def get_url(url):
return requests.get(url)The details dict carries target, args, kwargs, tries, elapsed, plus wait for on_backoff and value for on_predicate. Lists of handlers are accepted and called in order. Handlers run inside the except block, so sys.exc_info() and details['exception'] are both available.
Actually see the retrieslogging-setup
import logging
logging.getLogger('backoff').addHandler(logging.StreamHandler())
logging.getLogger('backoff').setLevel(logging.ERROR) # give-ups only
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
logger='my_logger', # or a Logger object, or None to silence it
)
def get_url(url):
return requests.get(url)The default 'backoff' logger has a NullHandler, so out of the box a service can retry for minutes without printing anything. INFO logs every retry; ERROR logs only give-ups.
Return None instead of raising after the last tryswallow-final-error
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_time=300,
raise_on_giveup=False,
giveup=fatal_code,
)
def get_url(url):
return requests.get(url)
result = get_url(url)
if result is None:
use_cached_value()The function returns None on give-up regardless of what the handlers do, so callers must check. This is the fastest way to turn an outage into silent data loss if you forget the None branch.
Read settings that only exist at runtimeruntime-configuration
def lookup_max_time():
return app.config["BACKOFF_MAX_TIME"]
@backoff.on_exception(backoff.expo, ValueError, max_time=lookup_max_time)
def do_work():
...Decorators run at import time, so a plain max_time=app.config[...] is read before your app is configured and freezes whatever value existed then. Pass a zero-argument callable and it is evaluated on each invocation.
Different policies for different failuresstacked-decorators
@backoff.on_predicate(backoff.fibo, max_value=13)
@backoff.on_exception(backoff.expo,
requests.exceptions.HTTPError,
max_time=60)
@backoff.on_exception(backoff.expo,
requests.exceptions.Timeout,
max_time=300)
def poll_for_message(queue):
return queue.get()Worst-case duration multiplies rather than adds: the outer predicate retry re-enters the inner exception retries every time. Work out the product before shipping this on a request path.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| tenacity | PyPI | You want the maintained general-purpose retry library, with composable stop/wait/retry policies and a Retrying object for retrying blocks of code |
| stamina | PyPI | You want opinionated production defaults on top of tenacity: sane caps, structured logging and metrics hooks, and a testing switch that turns retries off |
| urllib3 | PyPI | All you need is HTTP retries, where mounting a Retry policy on the adapter handles status codes, Retry-After, and connection errors below your code |