toolz
toolz is a pure-Python collection of about a hundred small functions for iterables, dictionaries, and other functions, borrowed from the list-processing traditions of Clojure and Underscore. It comes in three parts: itertoolz for sequences (groupby, unique, partition_all, sliding_window, pluck, frequencies, reduceby), dicttoolz for mappings (merge, merge_with, valmap, keyfilter, assoc, update_in, get_in), and functoolz for composing behaviour (curry, compose, pipe, memoize, juxt, excepts). Everything that touches a sequence is lazy and returns an iterator, and a parallel toolz.curried module gives you curried versions of the same functions so they can be stacked into a pipeline. It has no dependencies beyond the standard library and supports Python 3.9 and newer.
A well-made, tiny, dependency-free utility belt whose authors have publicly declared it finished and are keeping the lights on rather than developing it. Fine to depend on for what it does today, but treat the feature set as frozen and check the standard library first.
Use it if
- You write data transformation code where the same shapes keep repeating: group these records by a key, chunk this stream into batches of 500, walk this nested JSON safely, merge these config dicts with a rule for collisions
- You want that code lazy by default, so a groupby over a generator of a million rows does not first build a list of a million rows
- You already depend on toolz through dask, which requires it, so the import costs you nothing new
- You want dict transformations that return a new dict instead of mutating in place, which makes them safe to use inside a pipeline or a cached function
- You like point-free style and want pipe, compose_left, and the curried module rather than a stack of nested comprehensions
- You need an actively developed dependency. The README states directly that the project is alive but inactive, that the original maintainers have mostly moved on, that they commit only to critical bug fixes, Python version bumps, and security issues, and that they do not plan to spend much time reviewing contributions. The last push to master was 2026-01-01 and 88 issues are open. Nothing is going to be added for you
- You use a type checker. There is no py.typed marker and no stub files in the wheel, so mypy and pyright treat every toolz function as untyped and the return type of anything that passes through pipe or curry collapses to Any. Editor autocomplete inside a curried pipeline gives you nothing
- Half of what you want is already in the standard library. itertools has accumulate, pairwise, and batched, functools has cache and reduce, collections.Counter covers frequencies, and dict | dict has covered two-dict merge since Python 3.9. Adding a dependency for those is a bad trade
- The code is hot. These are Python-level wrapper functions, so pipe and curried composition add a call per element on top of what a plain comprehension costs. cytoolz is the drop-in Cython build for that case, and it needs a compiled wheel
- Your team does not already read functional Python. A five-stage compose_left of curried functions is dense to review, and when it raises, the traceback runs through toolz internals instead of pointing at the line that actually broke
- You expect curry to catch mistakes. Calling a three-argument curried function with two arguments does not raise TypeError; it quietly returns another curry object, so a missing argument travels downstream and fails somewhere unrelated
Setup reality
pip install toolz is genuinely a single step: pure Python, zero dependencies, no build stage, one wheel for every platform. What surprises people is everything after that. There are three import paths and they behave differently: toolz for the plain functions, toolz.curried for curried versions of the same names (including shadowing map, filter, and get), and tlz, a shim that transparently uses cytoolz when it is installed and falls back to toolz when it is not. Argument order is not uniform across the package. Sequence functions take the function first and the data last, as in groupby(key, seq) and valmap(func, d), because that is what makes currying work, while the dict-editing functions take the data first, as in assoc(d, key, value) and update_in(d, keys, func). Nearly every itertoolz function returns an iterator, so forgetting a list() around the result gives you an empty second pass rather than an error. The test suite ships inside the installed package, and there are no type annotations anywhere.
Patterns
Group records and count by keygroup-and-count
from toolz import groupby, countby, frequencies
rows = [
{"user": "ann", "status": "paid"},
{"user": "bob", "status": "open"},
{"user": "cat", "status": "paid"},
]
by_status = groupby(lambda r: r["status"], rows)
# {'paid': [{...}, {...}], 'open': [{...}]}
print(countby(lambda r: r["status"], rows)) # {'paid': 2, 'open': 1}
print(frequencies(["a", "b", "a"])) # {'a': 2, 'b': 1}groupby returns a real dict of lists, so it consumes the whole input; it is not the lazy itertools.groupby and it does not need the input sorted. countby is groupby plus len, and frequencies is Counter over the values themselves.
Build a pipeline with pipe and the curried modulecurried-pipeline
from toolz import pipe
from toolz.curried import filter, map, take
result = pipe(
range(1000),
filter(lambda n: n % 7 == 0),
map(lambda n: n * n),
take(5),
list,
)
# [0, 49, 196, 441, 784]toolz.curried deliberately shadows the builtin map, filter, and get, so import it under a namespace if that bothers you. Every stage stays lazy until list() runs, which means pipe over an open file reads only as many lines as take needs.
Read and update nested JSON without KeyErrornested-dict-access
from toolz import get_in, assoc_in, update_in
doc = {"user": {"profile": {"visits": 3}}}
get_in(["user", "profile", "visits"], doc) # 3
get_in(["user", "email"], doc, default="unknown") # 'unknown'
assoc_in(doc, ["user", "profile", "tier"], "pro")
update_in(doc, ["user", "profile", "visits"], lambda n: n + 1)get_in takes the key path first and the dict second; assoc_in and update_in take the dict first. That inconsistency is intentional (the first is curry-friendly, the others read like edits) and it is the most common source of TypeError here. All three return new structures and leave doc alone.
Merge dicts with a collision rulemerge-dicts
from toolz import merge, merge_with
merge({"a": 1, "b": 2}, {"b": 3}) # {'a': 1, 'b': 3}, last wins
merge_with(sum, {"a": 1}, {"a": 2, "b": 3}) # {'a': 3, 'b': 3}
merge_with(max, *daily_maxima)Plain merge is now covered by dict | dict on Python 3.9+, so it is only worth importing when you are merging a variable-length list of dicts. merge_with is the part with no standard-library equivalent: it hands every colliding key's values to your function as a list.
Map and filter over a dicttransform-dict
from toolz import valmap, keymap, itemmap, valfilter, keyfilter
prices = {"apple": 100, "pear": 250, "fig": 90}
valmap(lambda cents: cents / 100, prices)
keymap(str.upper, prices)
valfilter(lambda cents: cents > 95, prices)
keyfilter(lambda name: name.startswith("p"), prices)
itemmap(lambda kv: (kv[0].upper(), kv[1] * 2), prices)All five return a new dict of the same type as the input, so an OrderedDict or defaultdict in gives the same class out. itemmap passes the key/value pair as a single tuple, not as two arguments.
Batch a stream or slide a window over itchunk-and-window
from toolz import partition, partition_all, sliding_window
list(partition_all(2, [1, 2, 3, 4, 5])) # [(1, 2), (3, 4), (5,)]
list(partition(2, [1, 2, 3, 4, 5])) # [(1, 2), (3, 4)] last item dropped
list(sliding_window(2, [1, 2, 3])) # [(1, 2), (2, 3)]partition silently discards the incomplete tail unless you pass pad=; partition_all keeps it as a short tuple. Both are lazy, which is the point when you are batching database writes off a generator. On Python 3.12+ itertools.batched covers partition_all.
Cache a pure functionmemoize
from toolz import memoize
@memoize
def expensive(n):
return n * 2
cache = {}
@memoize(cache=cache, key=lambda args, kwargs: args[0])
def by_id(record):
return lookup(record)Unlike functools.lru_cache the cache is unbounded by default, so this leaks on unbounded input. Passing your own cache dict lets you inspect or clear it, and key lets you memoize on part of the arguments when the rest are unhashable.
Compose, fork, and negate functionscompose-functions
from toolz import compose, compose_left, juxt, complement
from toolz.curried import do
slugify = compose_left(str.strip, str.lower, lambda s: s.replace(" ", "-"))
same = compose(lambda s: s.replace(" ", "-"), str.lower, str.strip)
bounds = juxt(min, max)([3, 1, 9]) # (1, 9)
not_empty = complement(str.isspace)
traced = compose_left(do(print), str.upper) # print, then transformcompose applies right to left like mathematics, compose_left applies left to right like a pipeline; picking the wrong one produces a confusing TypeError deep inside the chain. do runs a side effect and returns its input unchanged, which is how you get logging into a pipeline without breaking it, but only the toolz.curried version takes one argument like that; the plain toolz.do wants the function and the value together.
Fold per group in one passreduce-by-key
from toolz import reduceby
orders = [
{"cust": "ann", "total": 30},
{"cust": "bob", "total": 12},
{"cust": "ann", "total": 20},
]
reduceby(
lambda o: o["cust"],
lambda acc, o: acc + o["total"],
orders,
0,
)
# {'ann': 50, 'bob': 12}This is the reason to reach for toolz over groupby plus a comprehension: it never materialises the per-group lists, so it runs over a stream in constant memory per key. The initial value is a plain object, so pass a factory-free immutable such as 0 or () and avoid a shared mutable default.
Work through a stream without loading itlazy-stream
from toolz import concat, mapcat, unique, pluck, take, drop, first
rows = concat(read_csv(p) for p in paths) # flatten one level
names = pluck("name", rows) # like operator.itemgetter, lazy
print(list(take(10, unique(names))))
words = mapcat(str.split, ["a b", "c d"]) # ['a', 'b', 'c', 'd']Everything here returns an iterator, so it can only be walked once. Calling list() twice on the same pluck result gives you the data and then an empty list, and that failure looks like a data bug rather than an iterator bug.
Turn an exception into a value inside a pipelineexcepts
from toolz import excepts
safe_int = excepts(ValueError, int, lambda e: None)
safe_int("12") # 12
safe_int("abc") # None
list(map(safe_int, ["1", "x", "3"])) # [1, None, 3]excepts only catches the exception classes you name, so a TypeError from passing None still propagates. The handler receives the exception object, which is worth using to log rather than swallowing failures silently across a whole column.
Use the compiled build when it is availableswap-in-cytoolz
# import from tlz instead of toolz
from tlz import groupby, pipe, curried
# tlz resolves to cytoolz when installed, otherwise plain toolz
# pip install cytoolz # optional, needs a compiled wheelThe tlz shim lets a library depend only on toolz while still getting the Cython speedup for users who install cytoolz. cytoolz tracks the same API but lags toolz releases and needs a wheel for your platform, so do not make it a hard requirement.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| cytoolz | PyPI | You want the identical API in a hot loop and can accept a compiled extension instead of pure Python |
| more-itertools | PyPI | You only need iterator recipes, want the same ideas with type annotations, and do not care about currying or dict helpers |
| funcy | PyPI | You want a similar utility belt with a more conventional Python feel and fewer functional-programming assumptions |