PyPIUtilsupdated 05 Aug 2026

msgpack

Item: msgpack
Rating: 4.3
Author: Keyoor

msgpack is the Python binding for MessagePack, a binary serialization format that carries roughly the same data shapes as JSON (maps, arrays, strings, bytes, ints, floats, booleans, null) but encodes them as compact bytes instead of text. You call msgpack.packb(obj) to get bytes and msgpack.unpackb(data) to get the object back. Because it has a real binary type, you can put raw bytes in a message without base64. The heavy lifting is a Cython extension module, so it is much faster than the pure Python json module, and the same format is readable from Go, Rust, Ruby, Java, and about fifty other languages. This is the package Celery, Redis clients, and many RPC layers reach for when JSON is too slow or too lossy.

msgpack · v1.2.1 · 66.9M/wk · ★ 2,102 · pushed 05 Aug 2026 · msgpack/msgpack-python · docs · reviewed by Keyoor

Verdict

The reference MessagePack implementation for Python, well maintained and fast enough that the format, not the library, is your bottleneck. Pick it when you need a binary interchange format across languages; pick msgspec instead if your messages have a known shape and you want validation and more speed in the same step.

API stability5/5packb, unpackb, Packer, and Unpacker have not changed shape since 1.0 in 2019; the only recent breaks were dropping Python 3.9 in 1.2.0 and deprecating the per-type max_str_len style limits in favor of max_buffer_size.

Docs3/5The README covers the common paths honestly, including the security options and the old-spec compatibility flags, but the option reference lives in docstrings and the README literally tells you to read them; the readthedocs site is thin autodoc.

Maintenance5/5Pushed August 2026 with 4 open issues (11 counting PRs); Inada Naoki has maintained it for over a decade and shipped 1.2.1 as a same-week fix for a segfault reported as GHSA-6v7p-g79w-8964.

Ecosystem4/5Around 67 million weekly downloads, pulled in by Celery, Redis tooling, and many RPC stacks, and MessagePack itself has implementations in dozens of languages; the direct-use tooling around this specific package is small because it does one job.

Use it if

You are moving many small messages between services or processes and JSON encode/decode is showing up in profiles: MessagePack is smaller on the wire and the C extension decodes faster
Your payloads contain raw bytes (images, hashes, pickled blobs, protobuf fragments) and you are tired of base64 padding them into JSON strings
You need cross-language interoperability without a schema file: any MessagePack implementation in any language can read what you wrote, unlike pickle
You want a streaming decoder that reads one message at a time from a socket or file, which Unpacker gives you with feed() or a file-like object

Skip it if

Your data is plain JSON-shaped and human debuggability matters: orjson decodes JSON at speeds close to msgpack, and you keep logs, curl output, and browser tooling that can read the payload
You want typed structs with validation and the last drop of speed: msgspec encodes and decodes MessagePack into declared Struct types faster than msgpack does into dicts, and it validates while decoding
You run on PyPy: the C extension (msgpack._cmsgpack) does not build there, so you fall back to msgpack.fallback, which is ordinary Python and much slower than PyPy's own JSON
You need self-describing, versioned messages with field names checked at the boundary: MessagePack maps are untyped, so a renamed key fails silently at the far end unless you build your own schema layer
You are on Windows without a matching wheel: the extension needs Visual Studio or the Windows SDK to compile, and the silent fallback to pure Python removes the reason you picked msgpack

Setup reality

pip install msgpack and you get prebuilt wheels for CPython on the usual platforms, so most people never compile anything. Where it gets annoying: if no wheel matches your platform, pip builds the Cython extension and you need a C toolchain, and if that fails you silently get msgpack.fallback, the pure Python implementation, which is slow enough to defeat the purpose. Check for msgpack._cmsgpack in a startup assertion if speed is the reason you installed this. Version 1.2 requires Python 3.10 or newer and dropped 3.9. The defaults also changed once, in 1.0: bytes now pack as the bin type, strings unpack as str, max_buffer_size defaults to 100 MiB, and strict_map_key rejects map keys that are not str or bytes. If you exchange data with an old service written against msgpack 0.x, you need use_bin_type=False when packing and raw=True when unpacking.

Patterns

Encode and decode a single messagepack-unpack-bytes

import msgpack

data = msgpack.packb({"id": 1, "tags": ["a", "b"]})
print(type(data), len(data))  # <class 'bytes'> 18

back = msgpack.unpackb(data)
print(back)  # {'id': 1, 'tags': ['a', 'b']}

dumps/loads are aliases of packb/unpackb for people porting from json or pickle. unpackb raises ExtraData if the buffer holds more than one message, which is the usual first surprise.

Read many messages from a streamstream-unpack

import msgpack

unpacker = msgpack.Unpacker()
while chunk := sock.recv(65536):
    unpacker.feed(chunk)
    for message in unpacker:
        handle(message)

feed() plus iteration is the correct socket pattern: the iterator stops when the buffer holds a partial message and resumes after the next feed. If unpack() raises anything other than OutOfData, throw the Unpacker away and build a new one; reuse after a failure is unsupported.

Stream to and from a file objectpack-to-file

import msgpack

with open("events.msgpack", "wb") as f:
    for event in events:
        msgpack.pack(event, f)

with open("events.msgpack", "rb") as f:
    for event in msgpack.Unpacker(f):
        handle(event)

pack/dump write to a file-like object; unpack/load read one message from one. For a file holding many messages use Unpacker(f), not repeated unpack() calls.

Round-trip timezone-aware datetimesdatetime-support

import datetime
import msgpack

now = datetime.datetime.now(datetime.timezone.utc)
packed = msgpack.packb({"created": now}, datetime=True)

back = msgpack.unpackb(packed, timestamp=3)
print(back["created"])  # datetime in UTC

datetime=True only accepts tz-aware datetimes; a naive one raises TypeError, which catches almost everybody once. The tzinfo is not preserved, only the instant, so you always get UTC back. timestamp= takes 0 for a Timestamp object, 1 for float seconds, 2 for int nanoseconds, 3 for datetime.

Serialize types msgpack does not knowcustom-types-hooks

import decimal
import msgpack

def default(obj):
    if isinstance(obj, decimal.Decimal):
        return {"__dec__": str(obj)}
    raise TypeError(f"Unknown type: {obj!r}")

def object_hook(obj):
    if "__dec__" in obj:
        return decimal.Decimal(obj["__dec__"])
    return obj

raw = msgpack.packb({"price": decimal.Decimal("1.10")}, default=default)
print(msgpack.unpackb(raw, object_hook=object_hook))

Always raise TypeError at the end of default() instead of returning obj; returning the original value makes the packer loop or emit garbage. object_hook fires for every map, so keep the check cheap.

Use the ext type for compact custom encodingsext-type

import array
import msgpack

def default(obj):
    if isinstance(obj, array.array) and obj.typecode == "d":
        return msgpack.ExtType(42, obj.tobytes())
    raise TypeError(f"Unknown type: {obj!r}")

def ext_hook(code, data):
    if code == 42:
        a = array.array("d")
        a.frombytes(data)
        return a
    return msgpack.ExtType(code, data)

values = array.array("d", [1.2, 3.4])
assert msgpack.unpackb(msgpack.packb(values, default=default), ext_hook=ext_hook) == values

ext beats a dict wrapper when size matters: one type byte plus the payload, no key strings. Codes 0 to 127 are yours; negative codes are reserved by the spec (-1 is the timestamp type).

Decode data from an untrusted sourceharden-untrusted-input

import msgpack

try:
    payload = msgpack.unpackb(
        body,
        max_buffer_size=1 * 1024 * 1024,
        strict_map_key=True,
    )
except (msgpack.exceptions.UnpackException, ValueError) as exc:
    reject(exc)

max_buffer_size defaults to 100 MiB and also caps preallocated container sizes, so a hostile header claiming a billion-element array cannot exhaust memory. strict_map_key defaults to True and limits map keys to str and bytes to avoid hash collision attacks; only turn it off for data you produced.

Decode arrays as tuples to cut allocationtuples-not-lists

import msgpack

packed = msgpack.packb([[1, 2], [3, 4]])
print(msgpack.unpackb(packed, use_list=False))
# ((1, 2), (3, 4))

Tuples are lighter than lists and this measurably helps on large payloads, per the project's own performance notes. The catch is that the decoded objects are immutable, so calling code that appends will break.

Reuse one Packer for many messagesreuse-packer

import msgpack

packer = msgpack.Packer(buf_size=1024 * 1024)
with open("out.msgpack", "wb") as f:
    for record in records:
        f.write(packer.pack(record))

A Packer keeps its internal buffer between calls, so reusing one avoids reallocating per message. The default buffer is 256 KiB; raise buf_size if single messages are larger. With autoreset=False the buffer accumulates and you call bytes(packer) yourself.

Walk a large array without materializing itread-container-headers

import msgpack

unpacker = msgpack.Unpacker(open("big.msgpack", "rb"))
count = unpacker.read_array_header()
for _ in range(count):
    row = unpacker.unpack()
    if row["kind"] != "click":
        continue
    handle(row)

read_array_header and read_map_header return the element count and let you unpack or skip() members one at a time, which keeps memory flat on multi-gigabyte files. skip() discards the next message without building Python objects.

Talk to a service still on the pre-1.0 formatlegacy-raw-format

import msgpack

# writing for an old peer that has no bin type
old = msgpack.packb([b"spam", "eggs"], use_bin_type=False)

# reading from one: everything string-ish arrives as bytes
print(msgpack.unpackb(old, raw=True))  # [b'spam', b'eggs']

This is the number one interop bug: msgpack 1.0 flipped the defaults to use_bin_type=True and raw=False, so a modern client and a 0.x server disagree about whether a field is str or bytes. Pin the flags explicitly at the boundary rather than guessing.

Fail loudly if you got the pure Python fallbackverify-c-extension

import msgpack

if msgpack._cmsgpack is None:  # AttributeError on builds without it
    raise RuntimeError("msgpack C extension missing; expect 10x slower encode")

# safer probe that works everywhere:
print(msgpack.Packer.__module__)  # 'msgpack._cmsgpack' when compiled

On PyPy, and on any platform where the wheel was missing and the build failed, msgpack quietly uses msgpack.fallback. Everything still works, just far slower, so assert this at startup if performance was the point.

Alternatives

Package	Registry	Pick it when
msgspec	PyPI	You want typed Struct classes, validation during decode, and the fastest MessagePack and JSON codecs available in Python.
orjson	PyPI	Your payloads are JSON-shaped and you would rather keep text you can read in logs than shave bytes off the wire.
ormsgpack	PyPI	You want a drop-in MessagePack codec written in Rust with built-in dataclass, datetime, UUID, and numpy handling.
cbor2	PyPI	You need an IETF-standardized binary format (RFC 8949) because a spec or certification requires it rather than a de facto one.