mrkeyoor.com_
Thu 06 Aug 07:43 UTC
PyPIWeb Backendupdated 06 Aug 2026

pyzmq

pyzmq is the Python binding for ZeroMQ, a C messaging library that gives you sockets with built-in patterns instead of raw byte streams. You create a Context, ask it for a socket of a given type, and bind or connect it over TCP, IPC, or in-process transports. The socket type decides the semantics: REQ and REP enforce a strict request then reply alternation, PUB and SUB fan out with topic filtering, PUSH and PULL round-robin work across workers, and DEALER and ROUTER give you fully asynchronous routing with explicit peer identities. Messages are whole frames, not streams, so you never write length-prefix framing yourself, and multipart messages let you build envelopes. There is no broker process and no server to run: ZeroMQ is a library that turns your own processes into the topology. pyzmq also ships an asyncio-aware layer, a Poller for multiplexing, CURVE encryption bindings, and a green variant for gevent.

Verdict

The right tool when you want socket-level messaging patterns between processes you own and no broker to operate. The cost is that reliability, retries, and back pressure become your protocol design work, so anything needing durable delivery belongs on a broker instead.

API stability5/5The Context and Socket API has been effectively frozen for years; the 27.0 release notes state outright that there are no breaking code or API changes and the only breaks are the manylinux and musllinux baselines used for Linux wheels
Docs4/5Sphinx docs on Read the Docs cover the full API, build options, and asyncio integration, and the separate ZeroMQ Guide has a Python version of every example; the gap is that the guide teaches ZeroMQ concepts rather than pyzmq specifics, so questions like linger and high water mark defaults take digging
Maintenance4/5Pushed within the last few days with a detailed changelog and steady releases through 27.1.0 in September 2025, mostly build system, wheel coverage, and typing work; effectively one primary maintainer, around 50 open issues, and the underlying libzmq has not cut a release since v4.3.5 in October 2023
Ecosystem4/5Around 25M downloads a week, largely because jupyter_client and ipykernel depend on it, so it is installed almost everywhere Python notebooks are; the pure ZeroMQ application community is much smaller than the download count suggests and most patterns come from the ZeroMQ Guide rather than Python-specific packages

Use it if

  • You need fast in-process, inter-process, or cross-host messaging between components you control, and running RabbitMQ or Kafka for that would be more infrastructure than the problem deserves
  • The messaging shapes you want are already socket types: fan-out with PUB and SUB, work distribution with PUSH and PULL, or asynchronous request routing with DEALER and ROUTER
  • You want message framing, automatic reconnection, and queueing handled for you rather than writing a length-prefix protocol over a raw TCP socket
  • You are building an asyncio service and want the same socket types inside it, which zmq.asyncio provides with awaitable send and recv
  • You need to talk to something already speaking ZeroMQ, which includes Jupyter kernels, many trading and robotics stacks, and a lot of scientific tooling
Skip it if

Setup reality

pip install pyzmq installs a wheel on macOS, Windows, Linux, and Android for CPython and PyPy, including a CPython 3.12 stable-ABI wheel since 27.0 that covers newer interpreters, plus free-threaded builds. That means no compiler for most people, and the wheel bundles libzmq so there is no system package to install first. Compiling is the awkward path: pip install --no-binary=pyzmq pyzmq needs a C compiler, cmake, and either a system libzmq or a bundled build that also fetches libsodium, and building from a git checkout additionally needs a recent Cython. What actually bites in production is not installation. Sockets are not thread-safe while the Context is, so each thread gets its own socket. Sockets default to a linger of -1, meaning context.term() blocks forever if anything is unsent, so set linger or use context.destroy(linger=0) at shutdown. Every socket has a send and receive high water mark that defaults to 1000 messages, and hitting it either blocks or, on PUB and SUB, silently drops. And recv_pyobj runs pickle on whatever arrives, which the pyzmq docs explicitly discourage without message authentication.

Patterns

REQ and REP for simple synchronous callsrequest-reply

# server.py
import zmq

with zmq.Context() as ctx, ctx.socket(zmq.REP) as sock:
    sock.bind('tcp://127.0.0.1:5555')
    while True:
        request = sock.recv_string()
        sock.send_string(f'echo: {request}')

# client.py
import zmq

with zmq.Context() as ctx, ctx.socket(zmq.REQ) as sock:
    sock.connect('tcp://127.0.0.1:5555')
    sock.send_string('hello')
    print(sock.recv_string())

REQ and REP enforce strict alternation: two sends in a row on a REQ socket raise ZMQError. If the peer dies mid-exchange the socket is stuck in the wrong state forever, which is why production code uses DEALER and ROUTER instead.

PUB and SUB with topic filteringpublish-subscribe

# publisher
import time, zmq

ctx = zmq.Context()
pub = ctx.socket(zmq.PUB)
pub.bind('tcp://*:5556')
time.sleep(0.2)  # let subscribers finish connecting
while True:
    pub.send_multipart([b'temps.kitchen', b'21.5'])
    time.sleep(1)

# subscriber
sub = ctx.socket(zmq.SUB)
sub.connect('tcp://127.0.0.1:5556')
sub.subscribe(b'temps.')      # prefix match; b'' means everything
while True:
    topic, payload = sub.recv_multipart()

A SUB socket with no subscribe call receives nothing at all. Filtering is a byte prefix, not a pattern. The sleep is the standard workaround for the slow joiner problem: messages published before the subscriber's handshake completes are simply gone.

Fan work out to a pool of workerspush-pull-pipeline

# ventilator
ctx = zmq.Context()
push = ctx.socket(zmq.PUSH)
push.bind('tcp://*:5557')
for i in range(1000):
    push.send_json({'task': i})

# worker (run several)
pull = ctx.socket(zmq.PULL)
pull.connect('tcp://127.0.0.1:5557')
while True:
    task = pull.recv_json()
    handle(task)

PUSH round-robins across connected PULL peers, so a slow worker still gets its equal share and holds up its own queue. Nothing is acknowledged: a worker that dies mid-task takes that task with it.

Asynchronous routing with explicit peer identitiesdealer-router

# broker side
router = ctx.socket(zmq.ROUTER)
router.bind('tcp://*:5558')

identity, empty, payload = router.recv_multipart()
router.send_multipart([identity, b'', b'ack:' + payload])

# client side
dealer = ctx.socket(zmq.DEALER)
dealer.setsockopt(zmq.IDENTITY, b'worker-7')
dealer.connect('tcp://127.0.0.1:5558')
dealer.send_multipart([b'', b'job-42'])
empty, reply = dealer.recv_multipart()

ROUTER prepends the sender's identity frame on receive and strips it on send, which is how replies find their way back. The empty delimiter frame is a convention inherited from REQ and REP; drop it and a REQ peer on the other end will not parse your reply.

Multiplex several sockets with a timeoutpoll-multiple-sockets

import zmq

poller = zmq.Poller()
poller.register(sub, zmq.POLLIN)
poller.register(pull, zmq.POLLIN)

while True:
    events = dict(poller.poll(timeout=1000))  # milliseconds
    if sub in events:
        handle_event(sub.recv_multipart())
    if pull in events:
        handle_task(pull.recv_json())
    if not events:
        heartbeat()

poll() takes milliseconds and returns an empty list on timeout, which is where periodic work like heartbeating belongs. A Poller can also register plain file descriptors, so you can mix ZeroMQ sockets with regular sockets in one loop.

Use ZeroMQ inside an asyncio serviceasyncio-sockets

import asyncio
import zmq
import zmq.asyncio

async def main():
    ctx = zmq.asyncio.Context()
    sock = ctx.socket(zmq.PULL)
    sock.bind('tcp://127.0.0.1:5559')
    try:
        while True:
            msg = await sock.recv_multipart()
            asyncio.create_task(handle(msg))
    finally:
        sock.close()
        ctx.term()

asyncio.run(main())

zmq.asyncio.Context is a different class from zmq.Context, and sockets made by one do not work with the other. The socket is still not safe to await from two tasks at once, so keep one reader per socket and hand work off with create_task.

Stop recv from blocking foreverreceive-timeout

import zmq

sock.setsockopt(zmq.RCVTIMEO, 2000)   # ms
sock.setsockopt(zmq.SNDTIMEO, 2000)

try:
    reply = sock.recv_string()
except zmq.Again:
    print('no reply within 2s')

# one-off non-blocking read
try:
    reply = sock.recv(zmq.NOBLOCK)
except zmq.Again:
    reply = None

Every blocking call defaults to waiting forever. A timeout surfaces as zmq.Again, which subclasses ZMQError with errno EAGAIN, so catch Again specifically rather than swallowing all ZMQError.

Shut down without hangingcontext-shutdown

import zmq

ctx = zmq.Context()
sock = ctx.socket(zmq.PUSH)
sock.setsockopt(zmq.LINGER, 1000)   # wait at most 1s for unsent messages
sock.connect('tcp://127.0.0.1:5557')
...
sock.close()
ctx.term()

# blunt version for shutdown paths: close every socket and drop pending sends
ctx.destroy(linger=0)

LINGER defaults to -1, meaning infinite, so ctx.term() with an unsent message and no reachable peer hangs your process at exit with no error. Setting LINGER on every socket or calling destroy(linger=0) is the fix.

Bound the queue instead of eating memoryhigh-water-mark

sock.setsockopt(zmq.SNDHWM, 10_000)
sock.setsockopt(zmq.RCVHWM, 10_000)

# detect back pressure instead of blocking
try:
    sock.send_json(payload, flags=zmq.NOBLOCK)
except zmq.Again:
    metrics.increment('zmq.backpressure')

The default is 1000 messages in each direction. When the send HWM is reached, PUSH and DEALER block while PUB silently discards, so a publisher outrunning its subscribers loses data with no exception anywhere. HWM must be set before bind or connect.

Send structured data safelyserialize-payloads

sock.send_json({'id': 7, 'ok': True})
msg = sock.recv_json()

# or bring your own codec
import msgpack
sock.send(msgpack.packb(obj, use_bin_type=True))
obj = msgpack.unpackb(sock.recv(), raw=False)

# avoid unless every peer is authenticated:
# sock.send_pyobj(obj); obj = sock.recv_pyobj()

recv_pyobj unpickles whatever bytes arrive, which is remote code execution if anything untrusted can reach the socket. The pyzmq docs discourage it without message authentication; send_json or msgpack costs nothing extra and does not have that property.

Encrypt and authenticate with CURVEcurve-encryption

import zmq

server_public, server_secret = zmq.curve_keypair()
client_public, client_secret = zmq.curve_keypair()

# server
rep = ctx.socket(zmq.REP)
rep.curve_secretkey = server_secret
rep.curve_publickey = server_public
rep.curve_server = True
rep.bind('tcp://*:5560')

# client
req = ctx.socket(zmq.REQ)
req.curve_secretkey = client_secret
req.curve_publickey = client_public
req.curve_serverkey = server_public
req.connect('tcp://127.0.0.1:5560')

Set every CURVE option before bind or connect. This encrypts the link and proves the server's identity, but it does not restrict which clients may connect: for that you run a zmq.auth authenticator with an allow list of client public keys.

Forward between two socketsproxy-device

import zmq

ctx = zmq.Context()
frontend = ctx.socket(zmq.XSUB)
frontend.bind('tcp://*:5561')   # publishers connect here
backend = ctx.socket(zmq.XPUB)
backend.bind('tcp://*:5562')    # subscribers connect here

try:
    zmq.proxy(frontend, backend)   # blocks forever
except KeyboardInterrupt:
    pass
finally:
    frontend.close()
    backend.close()
    ctx.term()

An XSUB and XPUB proxy gives publishers and subscribers one stable endpoint each so neither side needs to know the other's address. zmq.proxy never returns, so run it in its own process or thread, or use zmq.devices.ProcessDevice to get that for free.

Alternatives

PackageRegistryPick it when
pikaPyPIYou want a real broker with durable queues, acknowledgements, and routing rules, and can run RabbitMQ
nats-pyPyPIYou want brokered pub/sub and request/reply with a lightweight server and optional persistence through JetStream
pynngPyPIYou want the same scalability-protocol ideas from nanomsg-next-generation, with a cleaner story around thread safety and context objects