more-itertools
more-itertools is a pure-Python grab bag of functions for working with iterables, extending the standard library's itertools. It packages the recipes from the itertools docs (flatten, take, pairwise and friends) plus a few hundred more: chunked for fixed-size batches, windowed for sliding windows, peekable for lookahead, one and only for asserting cardinality, unique_everseen for order-preserving dedupe, map_reduce for group-and-aggregate, and long tails of combinatorics, math, and statistics helpers. Everything is lazy where it can be, has no dependencies, and works on any iterable, not just lists.
The standard companion to itertools and one of the safest dependencies on PyPI: pure Python, no deps, maintained for over a decade. Take it when you use several of its tools; skip it when the stdlib's batched and pairwise already cover your needs.
Use it if
- You keep rewriting the same five utilities (chunk a list, flatten one level, dedupe preserving order, sliding window, first item or default) and want tested, lazy implementations
- You process generators and streams where laziness matters: most functions consume iterables incrementally instead of materializing them
- You want defensive one-liners like one(), only(), and strictly_n() that turn silent cardinality bugs into immediate exceptions
- You need the itertools-docs recipes without copy-pasting them into every project, since they ship here maintained and importable
- You only need one or two functions: chunked is four lines over itertools.islice, batched has been in the stdlib since Python 3.12, and pairwise since 3.10, so a dependency may add nothing
- You are doing bulk numeric work: these are per-item Python-level loops, so NumPy or pandas will beat them by orders of magnitude on arrays
- Your team will not learn the vocabulary: with 200-plus functions, code reviews fill with 'what does ichunked/sieve/zip_broadcast do', and a plain loop is often clearer than an obscure import
- You want functional pipelines with currying and composition: this is a flat toolbox of functions, not a pipeline framework; toolz is built around that style
Setup reality
pip install more-itertools is as easy as Python packaging gets: pure Python, zero dependencies, type stubs included, Python 3.10+ required. The real costs come later. The API is enormous, so you will spend time in the docs discovering that the function you hand-rolled already exists under a name you did not guess (collapse vs flatten, divide vs distribute, spy vs peekable). Laziness cuts both ways: many results are one-shot iterators, so printing one for debugging consumes it, and functions like divide must materialize the input while distribute keeps it lazy but holds tee buffers. Major-version bumps do occasionally remove or rename long-deprecated functions, so pin accordingly.
Patterns
Break an iterable into fixed-size batcheschunk-into-batches
from more_itertools import chunked
for batch in chunked(user_ids, 100):
api.bulk_fetch(batch) # batch is a list of up to 100The last batch is shorter unless you pass strict=True, which raises instead. For huge chunks you cannot afford as lists, ichunked yields lazy sub-iterables.
Flatten one level or all levelsflatten-nested
from more_itertools import flatten, collapse
list(flatten([[0, 1], [2, 3]])) # [0, 1, 2, 3]
list(collapse([1, [2, [3, [4]]]])) # [1, 2, 3, 4]
list(collapse([['a', 'bc'], 'def'])) # ['a', 'bc', 'def']flatten removes exactly one level and will iterate strings into characters if they are the elements; collapse recurses to any depth but treats strings and bytes as atoms.
Slide a window across a sequencesliding-window
from more_itertools import windowed, sliding_window
list(windowed([1, 2, 3, 4], 3)) # [(1, 2, 3), (2, 3, 4)]
list(windowed([1, 2], 3, fillvalue=0)) # [(1, 2, 0)]
list(sliding_window([1, 2, 3, 4], 3)) # [(1, 2, 3), (2, 3, 4)]windowed pads short input with fillvalue and supports step; sliding_window is the leaner recipe that simply yields nothing when the input is shorter than the window.
Peek at an iterator without consuming itpeek-ahead
from more_itertools import peekable
lines = peekable(open('data.csv'))
if lines.peek('').startswith('#'):
next(lines) # skip header
for line in lines:
process(line)peek takes a default to avoid StopIteration on empty input. peekable also supports prepend() to push items back, and indexing to look further ahead at the cost of buffering.
Assert a query returns exactly one itemexactly-one-result
from more_itertools import one, only
user = one(u for u in users if u.id == user_id)
# raises ValueError if zero or more than one match
maybe = only(matches, default=None)
# returns the item, the default when empty, raises when 2+These replace the [0]-and-hope pattern; the failure mode becomes a clear exception at the query site instead of a wrong result downstream.
Get the first item or a defaultfirst-with-default
from more_itertools import first, last, nth
first([], 'fallback') # 'fallback'
first(results) # raises ValueError if empty, no default
last(open('log.txt')) # final line
nth((x*x for x in range(10)), 3) # 9first(iterable, default) is the cleanest replacement for next(iter(x), default). last must consume the whole iterator unless the input supports reversed().
Deduplicate while keeping orderdedupe-preserve-order
from more_itertools import unique_everseen
list(unique_everseen('AAAABBBCCDA')) # ['A', 'B', 'C', 'D']
list(unique_everseen(rows, key=lambda r: r.id)) # first row per idKeeps a seen-set, so memory grows with distinct keys; unhashable elements fall back to a slow list scan. unique_justseen only collapses consecutive duplicates and stays O(1) memory.
Split items into false and true groupspartition-by-predicate
from more_itertools import partition
odds, evens = partition(lambda n: n % 2 == 0, range(10))
list(odds) # [1, 3, 5, 7, 9]
list(evens) # [0, 2, 4, 6, 8]Returns false-items first, which trips everyone up. Both outputs share a tee buffer, so consuming one far ahead of the other buffers the difference in memory.
Group by key and reduce in one passgroup-and-aggregate
from more_itertools import map_reduce
orders = [('eu', 30), ('us', 50), ('eu', 20)]
totals = map_reduce(
orders,
keyfunc=lambda o: o[0],
valuefunc=lambda o: o[1],
reducefunc=sum,
)
# {'eu': 50, 'us': 80}Unlike itertools.groupby this does not require sorted input; it builds a dict of all groups in memory, returning a defaultdict-like mapping.
Split an iterable where a condition holdssplit-on-condition
from more_itertools import split_at, split_when
list(split_at('one\ntwo\n\nthree'.split('\n'), lambda l: l == ''))
# [['one', 'two'], ['three']]
list(split_when([1, 2, 3, 3, 2, 5], lambda a, b: a > b))
# [[1, 2, 3, 3], [2, 5]]split_at drops the separator elements (keep_separator=True keeps them as their own groups); split_when compares adjacent pairs, handy for splitting on descents or time gaps.
Deal items into n groupsdivide-work-evenly
from more_itertools import divide, distribute
groups = [list(g) for g in divide(3, range(10))]
# [[0, 1, 2, 3], [4, 5, 6], [7, 8, 9]] contiguous runs
shards = [list(s) for s in distribute(3, range(10))]
# [[0, 3, 6, 9], [1, 4, 7], [2, 5, 8]] round-robindivide needs the full input in memory (it materializes to know the length); distribute stays lazy but uses tee, so consuming shards unevenly buffers. Pick by whether order runs must stay contiguous.
Count items in a generatorcount-generator-items
from more_itertools import ilen
error_count = ilen(l for l in open('app.log') if 'ERROR' in l)len() does not work on generators; ilen consumes the iterator to count, so it is single-use and O(n), which is still better than building a throwaway list.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| toolz | PyPI | You want a functional style with curried, composable pipelines over dicts and iterables rather than a flat utility box. |
| cytoolz | PyPI | You want the toolz API with Cython speed for hot per-item code paths. |
| boltons | PyPI | You want general stdlib gap-fillers (dicts, files, queues, iteration) in one dependency instead of an iterables specialist. |