more-itertools review
more-itertools 11.1.0 fills gaps around Python's itertools with named operations for chunking, lookahead, windows, splitting, deduplication, cardinality checks, combinatorics, and running statistics. Most functions accept ordinary synchronous iterables, so generator pipelines can stay lazy where that function's contract permits. The 11.1 release fixes empty numeric_range results and peekable typing, reduces islice_extended memory use, forwards generator control methods through serialize, and lets seekable index values it has already cached.
more-itertools 11.1.0 installed in 0.2 seconds, occupied 1 MB, and produced 0 audit findings in our sandbox, making it cheap for Python 3.10+ projects that reuse several iterator recipes. Leave it out when itertools already has the function, the source is async, or the selected helper's caching can grow with the input.
We installed it
| Install | ✓ · 0.2s | 1 package on disk · 1 MB |
| Import | ✓ | import more_itertools in 0.14s · pure Python · py.typed · requires Python >=3.10 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does more-itertools install cleanly?
Yes. In a fresh container with an empty cache, pip install more-itertools finished in 0.2s, leaving 1 package and 1 MB on disk. pip-audit reported no known vulnerabilities.
What does more-itertools need to run?
Python >=3.10, and nothing compiled: it is pure Python. In our run import more_itertools succeeded in 0.14s, and the package ships py.typed for type checkers.
more-itertools or toolz: which should you use?
toolz: Choose it when iterator transforms belong in a functional pipeline built around compose, curry, and collection helpers. more-itertools 11.1.0 installed in 0.2 seconds, occupied 1 MB, and produced 0 audit findings in our sandbox, making it cheap for Python 3.10+ projects that reuse several iterator recipes.
When should you not use more-itertools?
Python's itertools already has the operation; pairwise and batched now live in the standard library, and more-itertools has deprecated its pairwise alias
Discussed on
- hnMore Itertools206 points
- hnMore-itertools: More Python routines beyond itertools5 points
- hnMore itertools (Python)3 points
- hnMore Itertools3 points
Use it if
- Your code repeatedly implements chunking, peeking, windowing, splitting, or exactly-one checks around generators
- You need a documented iterator recipe whose consumption and edge cases are easier to review than a local generator
- Inputs may be large enough that keeping an operation lazy matters, and the chosen helper documents bounded consumption
- The project runs Python 3.10 or newer and a dependency-free utility package fits its policy
- Python's itertools already has the operation; pairwise and batched now live in the standard library, and more-itertools has deprecated its pairwise alias
- The application accepts async iterables; these helpers use the synchronous iterator protocol and cannot await an async generator
- Input is unbounded and the helper retains history; unique_everseen stores every distinct key, while seekable and peekable cache values to support their extra behavior
- Consumers will read tee-derived branches at very different speeds; partition, unzip, distribute, and related fan-out helpers can buffer items for the lagging branch
- You depend on Python 3.9 or older, or on removed version 10 behavior; major 11 drops Python 3.9 and makes unique_everseen reject unhashable input without the former fallback
Setup reality
Our Python 3.12 install of more-itertools 11.1.0 succeeded in 0.2 seconds. It left 1 package and 1 MB on disk, with 0 direct dependencies and 0 known pip-audit findings. The distribution is pure Python, requires Python 3.10 or newer, and includes py.typed. import more_itertools completed in 0.14 seconds. Our installed metadata did not identify a license, although the current PyPI record declares an MIT license expression.
No service, credential, environment variable, or config file appears after installation. Public helpers import from more_itertools. Their names can hide different return and storage choices: chunked emits lists, ichunked emits child iterators, and divide first turns its source into a sequence. Read the individual function notes before feeding one a file stream or generator whose size is unknown.
Laziness does not guarantee constant memory. unique_everseen keeps a set of all observed keys; seekable stores consumed values for rewinding; peekable fills a cache when asked to look ahead. partition and unzip return linked iterators, so reading one branch far ahead holds values needed by the others. Version 11 changed unique_everseen to raise TypeError for unhashable elements unless you supply a hashable key.
All APIs here consume synchronous iterables. Version 11 adds concurrent_tee and locking wrappers such as serialize for access from threads, but those tools do not turn blocking work into async I/O. Cardinality helpers also consume input as part of their answer: one raises for both 0 and multiple items, only can supply a default for 0, and both must examine enough values to prove that a second match does not exist.
Patterns
Yield list batches chunk-fixed-size
from more_itertools import chunked
for batch in chunked(record_ids, 100):
send_batch(batch)chunked yields lists, and the final list may contain fewer than 100 items. Pass strict=True when a short final batch should raise ValueError.
Yield lazy child batches chunk-child-iterators
from more_itertools import ichunked
for batch in ichunked(rows, 1000):
consume(batch)ichunked returns sub-iterators. Consume them in order because requesting later chunks first forces caching for earlier, unfinished chunks.
Inspect the next item peek-next-value
from more_itertools import peekable
lines = peekable(source)
if lines.peek(None) == 'HEADER':
next(lines)peek(None) returns the default on empty input. Looking several positions ahead stores those values in the peekable cache.
Read overlapping windows slide-complete-windows
from more_itertools import sliding_window
for previous, current, following in sliding_window(values, 3):
inspect(previous, current, following)sliding_window emits only complete windows. Use windowed when padding a short input or moving by a step other than 1 is part of the result.
Cut a stream at blank lines split-at-separator
from more_itertools import split_at
sections = split_at(lines, lambda line: line.strip() == '')
for section in sections:
process(section)split_at drops separator items unless keep_separator is set. Consecutive matches may create empty sections, depending on the options.
Require exactly one result enforce-single-match
from more_itertools import one
account = one(
account for account in accounts
if account.external_id == wanted_id
)one raises ValueError when it sees 0 or more than 1 item. The function consumes enough input to determine that no second result exists.
Keep the first row for each ID dedupe-by-key
from more_itertools import unique_everseen
unique_rows = unique_everseen(rows, key=lambda row: row.id)
for row in unique_rows:
write(row)The internal seen set grows with each distinct key. Version 11 raises TypeError for unhashable elements when no hashable key function is supplied.
Split rejected and accepted items partition-by-predicate
from more_itertools import partition
rejected, accepted = partition(is_valid, records)
store_rejected(rejected)
store_accepted(accepted)partition returns the false branch first. Both outputs share the source, so a fast consumer can cause buffering for the slower branch.
Flatten nested batches once flatten-one-level
from more_itertools import flatten
values = list(flatten([[1, 2], [3], [4, 5]]))flatten removes exactly one nesting level. collapse handles recursive nesting and treats strings and bytes as atomic by default.
Count a filtered iterator count-consumed-items
from more_itertools import ilen
error_count = ilen(
line for line in log_lines
if 'ERROR' in line
)ilen avoids creating a list but still reads the entire source. log_lines is exhausted when the count returns.
Rewind over cached input remember-and-seek
from more_itertools import seekable
items = seekable(source)
first = next(items)
second = next(items)
items.seek(0)
assert next(items) == firstseekable can revisit values it has already cached. It does not provide direct access to untouched positions without consuming up to them.
Calculate a rolling mean read-running-mean
from more_itertools import running_mean
for average in running_mean(samples, window_size=5):
emit(average)A window_size produces moving results over that many values. Check the documented output length before aligning averages with original timestamps.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| toolz | PyPI | Choose it when iterator transforms belong in a functional pipeline built around compose, curry, and collection helpers |
| boltons | PyPI | Choose it when you also need caches, file utilities, URL handling, sockets, and other additions beyond iteration |
| iteration-utilities | PyPI | Choose it when its Cython-backed iterator types and specific operator set match the workload |
More utils guides
lru-cache · type-fest · ajv · p-limit · find-up · js-yaml · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.