mrkeyoor.comAsk

jundot/omlxtested 13 Oct 2026commit b7f366b

omlx review

Evaluate oMLX on a supported Apple Silicon Mac if its model controls and persistent prompt cache address a problem in your workflow.

  • Build 31 s
  • Tests failed
141 sto install
  • 181 packages
LLM Toolsevaluationupdated 08 Nov 2026
+137stars / 7d

oMLX is a local model server for Apple Silicon Macs, with a menu-bar app and browser dashboard. Its README documents model pinning, automatic eviction and a RAM-and-SSD prompt cache for reusing matching context. Our recorded run did not validate model serving.

Verdict

Evaluate oMLX on a supported Apple Silicon Mac if its model controls and persistent prompt cache address a problem in your workflow. Our October 13, 2026 run at b7f366b recorded successful install and build steps, but tests stopped at a libmlx.so import failure after 17 seconds. It establishes no inference speed, cache benefit or client compatibility. Start with the documented DMG and validate your model and client before relying on it for coding work.

We ran it

Lab card: what happened when we ran omlx
Lab card from our sandbox run
Screenshot of omlx (omlx.ai)
omlx.ai
Install✓ · 141s181 packages · 6249 MB
Build✓ · 31s
Tests✗ · 17sran, no count parsed
Known vulns0(pip-audit)
Repo1500 files~688,899 lines of source · 135.7 MB · 5 CI workflows · tests dir

Answers from our run

Does omlx build from source?

Dependencies installed in 141 seconds (181 packages), and the build succeeded in 31 seconds. We cloned commit b7f366b into a clean Debian container with 3 CPUs and no project-specific setup.

Do omlx's tests pass?

No. The project's own test command exited unsuccessfully after 17 seconds, and its output did not expose an individual test count.

Does omlx have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use omlx?

Linux, Windows or Intel Mac deployments: the documented requirements specify Apple Silicon and macOS 15.0 or later.

What are the alternatives to omlx?

Ollama, llama.cpp. Evaluate oMLX on a supported Apple Silicon Mac if its model controls and persistent prompt cache address a problem in your workflow.

Setup3/5Our source checkout installed in 141 s and built in 31 s, a heavy install of 181 dependencies, and we did not run a README task.Editor’s note: The sandbox completed package steps, but installation and application launch were not validated on a supported Apple Silicon Mac.
Docs4/5The supplied README gives concrete installation routes, kernel prerequisites, cache behavior, model controls and tool-template requirements. The body explains the five-point rubric; supported-route execution remains unverified.
Community5/5Last push on 2026-11-08 and 23,640 stars.Editor’s note: The supplied issue list shows reporting activity, but support responsiveness, maintainer capacity and adoption were not assessed.
Maturity2/5Latest release v0.7.4.dev1 on 2026-11-04 (pre-1.0), repository created 2026-02-13 and its test suite did not pass in our sandbox.Editor’s note: The recorded run did not establish supported-Mac serving reliability, cache recovery or sustained client compatibility.

How we score

Discussed on

  1. hnShow HN: oMLX – SSD-backed KV cache cuts coding agent TTFT from 90s to 1s on Mac4 points
  2. hnOMLX: Local LLM Server for Apple Silicon Macs4 points
  3. hnShow HN: oMLX – coding agents on local LLMs without the painful reprefill3 points
  4. hnOMLX – LLM Inference Server for Apple Silicon (Ollama for MLX)3 points

Who it’s for

Apple Silicon developers who want dashboard controls for keeping everyday models loaded and unloading others.
Coding-agent users willing to validate their model's tool-call template and client behavior.
Mac users evaluating persistent prompt caching for conversations with repeated prefixes.

Who it’s NOT for

Linux, Windows or Intel Mac deployments: the documented requirements specify Apple Silicon and macOS 15.0 or later.
Users who need this review to establish inference speed or supported-Mac reliability: neither was measured.
Source builders expecting Command Line Tools alone to compile native custom kernels: the README requires full Xcode with the Metal toolchain.
Teams requiring established multi-Mac operation: the README labels that feature experimental.

Setup reality

Our October 13, 2026 sandbox run at commit b7f366b completed installation in 141 seconds and the build step in 31 seconds. It recorded 181 dependencies and 6,249 MB on disk. Tests exited 4 after 17 seconds while importing mlx.core because the loader could not open libmlx.so. No passing-test count or completed suite result is established.

pip-audit recorded an advisory total of 0 and a package count of 0. Its coverage is unknown, so that result does not establish that the installed dependencies were meaningfully audited or are safe.

The recorded configuration was 3 CPUs, 8 GB RAM and image ghcr.io/astral-sh/uv:python3.12-bookworm. The image label does not establish actual runtime versions or the operating system. The dependency storage figure is not a Mac installation requirement.

The README requires Apple Silicon, macOS 15.0 or later and Python 3.11 through 3.13. The official DMG includes precompiled native custom kernels; compiling them requires full Xcode with the Metal toolchain. Our run did not launch the Mac application, serve a model or exercise client integrations.

The Mac controls are the reason to try oMLX

The oMLX README documents model pinning to keep frequently used models loaded, automatic eviction when memory runs low, and manual loading and unloading through the dashboard. Those controls give Mac developers a concrete reason to evaluate it when several local models share one machine.

The README also documents text and vision models, embeddings, rerankers, and OpenAI- and Anthropic-compatible APIs. These are project claims about supported capabilities. Our run did not reach a model conversation, so client compatibility and inference performance remain unverified here.

Consider a trial if the model controls address a recurring problem in your workflow. This review supplies no measured speed advantage over another runner.

What happened when we ran it

Our recorded sandbox run took place on October 13, 2026, at commit b7f366b. Installation succeeded in 141 seconds, and the build step succeeded in 31 seconds. The record lists 181 dependencies and 6,249 MB on disk. Those successful steps do not establish that a working Mac application or model server was installed.

Tests exited with code 4 after 17 seconds. While loading tests/conftest.py, the command imported an oMLX workaround module, which imported mlx.core. The loader reported ImportError: libmlx.so: cannot open shared object file: No such file or directory. The log establishes that import failure, but not its cause or whether the suite would pass on a supported Mac. No passing-test count is available.

The sandbox configuration was 3 CPUs, 8 GB RAM and image ghcr.io/astral-sh/uv:python3.12-bookworm. That image label does not establish the actual runtime versions or operating system. The recorded dependency footprint is not a minimum Mac storage specification.

pip-audit recorded an advisory total of 0 and a package count of 0. Coverage is unknown; the zero advisory total does not establish a meaningful audit of the installed dependencies. No new execution was performed for this correction.

Choose an installation route with the kernel requirements in mind

The documented requirements are Apple Silicon, macOS 15.0 or later and Python 3.11 through 3.13. The official DMG includes precompiled native custom kernels. Compiling those kernels requires full Xcode with the Metal toolchain; Command Line Tools alone are insufficient.

The README distinguishes installations that build kernels from routes that omit them. The source route uses make install to include them, while make install-no-kernels omits them. A plain editable pip installation also skips them. Homebrew builds them with --with-custom-kernel, which requires full Xcode. A successful Python installation therefore does not establish native-kernel availability.

For an initial trial without a kernel build environment, start with the documented DMG. The app's Welcome screen covers the model directory, server start and first model download. Confirm that your chosen model loads and your client completes a conversation before adding optional MCP support or experimenting with multiple Macs. These are proposed acceptance checks, not results from our run.

Repeated prompt prefixes are the cache's intended workload

According to the README, frequently accessed cache blocks stay in RAM, while blocks can move to SSD when the hot cache fills. A later request with a matching prefix can restore them from disk, including after a server restart.

That mechanism makes repeated opening context a useful workload to evaluate. Our run measured no cache hits, restart recovery or time to first token. On the intended Mac, compare a fresh conversation with one that reuses the same prefix while keeping the model settings fixed. The documentation supports that evaluation design; it supplies no measured saving for your workload.

Model pinning and per-model idle timeouts answer a separate question: which models should occupy memory between requests? Identify the model you need immediately available before deciding what to pin.

Tool calls need a compatible model template

The README requires the model's chat template to support the tools parameter. An endpoint accepting a request does not establish that the selected model can complete an agent's tool workflow. Check a tool call and its follow-up response with the client you intend to use.

The README labels multi-Mac inference experimental, so keep it outside the first acceptance trial unless distributed operation is the feature you need to evaluate. Support responsiveness and sustained reliability were not assessed. The supplied issue list does not establish reproduced defects or support quality.

The reviewed README and the recorded execution also have different scopes: the execution is tied to b7f366b; this review does not establish that every capability in the supplied README existed at that commit.

When an alternative fits better

Ollama documents macOS, Windows and Linux installation routes. Ollama has a REST API for running and managing models. Consider it when documented installation across those operating systems is your immediate requirement.

llama.cpp documents C/C++ inference and multiple hardware backends. llama.cpp documents an OpenAI-compatible API server. Consider it when backend selection or the C/C++ implementation matters to your deployment. Neither alternative was benchmarked against oMLX here.

How the documentation score works

Documentation receives 4/5 for its concrete installation routes, kernel prerequisites, cache explanation, model controls and tool-template constraint. This is an editorial assessment of the supplied documentation. On this scale, 1 means essential guidance is missing, 3 means the main route is explained with meaningful gaps, and 5 means instructions are corroborated across the reviewed routes; 2 and 4 sit between those levels. The supplied guidance is detailed, but its supported installation routes were not executed in our run.

Setup ease, community and maturity remain unassessed because supported-Mac installation, support responsiveness and sustained serving reliability were not established. Before replacing an existing runner, make oMLX handle your chosen model, client and repeated-prefix conversation on the Mac that will host it.

Alternatives

ProjectWhat it isPick it when
Ollama gh↗A model runner with documented macOS, Windows and Linux installation routes and a REST API.Consider Ollama when documented installation across those operating systems is the immediate requirement.
llama.cpp gh↗C/C++ model inference with multiple hardware backends and an OpenAI-compatible API server.Consider llama.cpp when backend selection or its C/C++ implementation matters to your deployment.

Sources

  1. oMLX README
  2. Ollama README
  3. llama.cpp README
  4. oMLX public issues

More llm tools reviews