The rare case where pooling Macs makes sense
Most local model runners assume one computer owns the weights and performs every operation. exo targets the harder case: the model is larger than one machine's available memory. It discovers other exo nodes, measures the topology, and places model shards across them. The cluster can use pipeline or tensor parallelism, while each node exposes the same dashboard and API.
That idea is particularly attractive on Apple Silicon because CPU and GPU share a large unified memory pool. Several high-memory Mac Studios can collectively hold models that would otherwise require specialized accelerator servers. exo uses MLX for inference and distributed communication, with an RDMA path over Thunderbolt 5 for lower-latency links. The README shows clusters running very large Qwen, DeepSeek, and Kimi models, and supplies a benchmark utility so owners can compare placements on their own hardware.
The surrounding interface is better than a science demo. A web dashboard shows nodes and instances, downloads models, and hosts chat. API compatibility covers OpenAI Chat Completions, OpenAI Responses, Claude Messages, and Ollama formats. Users can preview valid placements, create an instance asynchronously, wait for readiness through server-sent events, run inference, and delete the instance. Offline mode, multiple model directories, read-only shared storage, and optional image support make the system adaptable to a lab.
Apple Silicon is the product, not one backend
The phrase “all your devices” needs qualification. The current platform document names a small group of Apple Silicon Macs as tier-one tested hardware. The README says macOS uses its GPU, while Linux currently runs on CPU and Linux accelerator support is under development. The platform roadmap lists Linux CUDA as planned and Windows later. Code for other paths and community pull requests may exist, but that is not the same as maintained support.
The easiest installation is the macOS app through a DMG or Homebrew cask. It requires macOS Tahoe 26.2 or newer, runs in the background, asks for permission to change system settings, and installs a network profile. That is reasonable for a dedicated cluster node, but unusually invasive for a laptop utility. Uninstallation has its own cleanup flow for the launch daemon, scripts, logs, and network location.
Running from source is far heavier. The documented path needs Xcode, Homebrew, uv, Node, nightly Rust, and a pinned fork of macmon, followed by an npm dashboard build and uv run exo. Nix offers a shorter command, but accepting its binary cache still requires system configuration. The project also pins custom MLX and MLX-LM forks, a sign that its distributed behavior depends on a fast-moving stack rather than ordinary released Python packages.
RDMA is powerful and operationally awkward
Thunderbolt RDMA is the feature that makes a Mac cluster more than a collection of slow networked workers. Enabling it requires booting each Mac into Recovery mode and running rdma_ctl enable. Every participating device needs a direct connection to every other device, the cables must support Thunderbolt 5, one Mac Studio port is excluded, and the machines must run exactly matching macOS versions. The source setup script disables Thunderbolt Bridge and reconfigures RDMA ports for DHCP.
Those requirements become cumbersome as the cluster grows because a full mesh consumes ports and cables rapidly. More importantly, recovery is not yet appliance-grade. Issue 1847 describes a three-node M3 Ultra deployment suffering repeated JACCL crashes during normal inference, sometimes leaving the cluster unable to recover without rebooting every node. Comments document related worker crashes that leave an apparently alive process detached from useful RDMA work. A separate August report says the macOS app's memory use grows even with no model loaded and requires periodic restarts.
Node identity is also deliberately nonpersistent. Issue 2248 traces fresh IDs on restart to code that bypasses the old disk-persistence path. In a multi-node cluster, a restart can therefore look like a new computer, producing temporary placement failures and instance eviction while topology settles. These are manageable problems in a research lab with an operator present. They are bad properties for unattended serving.
API compatibility has important edges
Having four familiar API shapes is useful, but compatibility is not equivalence. Issue 2193 shows Claude Code sending a system-role entry inside its message list and receiving a 422 response. A community pull request adds lenient handling, but it remained unmerged. Issue 2233 identifies a generic tool-call parser that misses markers when streaming chunks split them at inconvenient boundaries, turning structured calls into ordinary text. Test the exact client, model, prompt template, and tools you plan to use.
Long prompts and concurrency reveal deeper distributed risks. Issue 2208 documents large agent prompts hanging or returning garbled output during two-node tensor and RDMA work, while short dashboard prompts succeed. Issue 2108 reports pipeline inference deadlocking at 32 concurrent requests because ranks reach a blocking collective at different times. Recovery requires restarting both nodes. These reports are specific configurations, not proof that every cluster fails, but they rule out assuming production readiness from a successful single chat.
Health and the buying decision
The maintenance signal is currently concerning. The last push to the repository was June 23, 2026, and release 1.0.71 shipped April 23. GitHub counted 334 open issues and pull requests together, including 188 actual open issues. Users and contributors were still filing detailed reports and opening fixes in August, so the community is active. However, an August issue asking whether maintainers still watch the project had no maintainer response, while contributors discussed consolidating work in a fork. This is not enough evidence to call exo abandoned, but it warrants caution.
exo remains compelling for people who already own several compatible Macs and want to explore huge local models. Begin with two nodes, benchmark the real prompt length, exercise restarts, and document full-cluster recovery. For a single machine, Ollama or llama.cpp is simpler. For a Linux GPU service with uptime and concurrency requirements, vLLM is the more sensible foundation today.