mrkeyoor.comAsk

JustVugg/colibritested 13 Oct 2026commit bf24429

colibri review

Our Colibri run installed 36 packages in 28 seconds and built in 3 seconds, but skipped tests and did not establish inference performance.

  • Build 3 s
28 sto install
  • 36 packages
LLM Toolsevaluationupdated 14 Oct 2026
+1,125stars / 7d

Colibri runs open AI models locally by keeping frequently used weights in memory and reading other weights from disk as needed. It gives you a chat interface and APIs for existing apps, with a C inference engine underneath the Python setup tools.

Verdict

Our Colibri run installed 36 packages in 28 seconds and built in 3 seconds, but skipped tests and did not establish inference performance. Try it for local model experiments where storage can compensate for limited memory and you can investigate engine-specific failures. I would require a successful workload on the exact model and hardware before putting it behind an unattended application.

We ran it

Lab card: what happened when we ran colibri
Lab card from our sandbox run
Screenshot of colibri (justvugg.github.io/colibri)
justvugg.github.io/colibri
Install✓ · 28s36 packages · 38 MB
Build✓ · 3s
Testsn/ano test script
Known vulns0(pip-audit)
Repo1191 files~342,274 lines of source · 30.9 MB · 4 CI workflows

Answers from our run

Does colibri build from source?

Dependencies installed in 28 seconds (36 packages), and the build succeeded in 3 seconds. We cloned commit bf24429 into a clean Debian container with 3 CPUs and no project-specific setup.

Does colibri have tests you can run?

Not through a standard command: the project exposes no test script or target that our harness could run.

Does colibri have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use colibri?

Operators needing predictable interactive speed on slow storage: the README says disk speed governs the large streamed models.

What are the alternatives to colibri?

llama.cpp, Ollama, vLLM. Our Colibri run installed 36 packages in 28 seconds and built in 3 seconds, but skipped tests and did not establish inference performance.

Setup3/5Package checks pass, but model storage and backends need attention
Docs4/5Detailed model, platform, API, and troubleshooting guidance
Community4/5October 17 push and current hardware reports show active use
Maturity3/5Active releases, skipped harness tests, and an exact-commit crash report

Discussed on

  1. hnShow HN: Getting GLM 5.2 running on my slow computer937 points
  2. hnShow HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri48 points

Who it’s for

Local inference experimenters with fast storage who want to try models larger than their available memory.
Developers who want to inspect expert routing and adjust CPU or GPU placement.
Technical users connecting existing OpenAI-compatible or Anthropic-compatible clients to local models.

Who it’s NOT for

Operators needing predictable interactive speed on slow storage: the README says disk speed governs the large streamed models.
Teams expecting identical features across engines: Inkling and OLMoE lack tool calling, according to the API table.
Buyers treating a package build as inference validation: our harness skipped tests, and issue #2132 reports an Inkling crash at our exact commit.
Mac users expecting the automatic setup to select Metal: the README says that setup builds for CPU and describes a separate Metal path.

Setup reality

Our sandbox installed 36 Python packages in 28 seconds, using 38 MB, then built successfully in 3 seconds at commit bf24429 on October 13, 2026. Tests were skipped because the harness found no script or target. Pip-audit reported 0 known vulnerabilities.

Useful inference also requires model weights, enough storage, and a suitable memory budget. The README documents a local setup wizard and compiler packages for Debian, plus optional CUDA or Vulkan tooling. A paid model API is not part of the local quick start.

The 8 GB sandbox did not establish model speed or GPU compatibility. The README warns against storing models under WSL's /mnt/c, and automatic macOS setup uses CPU rather than Metal. Our Python ecosystem classification describes the harness path, while GitHub identifies C as the primary language.

Disk access makes larger local models possible

Choose Colibri when the model you want exceeds your fast memory and you are willing to spend storage bandwidth to run it. The engine keeps the always-needed weights in RAM, caches frequently used experts, and fetches other experts from disk. That gives a technically curious owner a way to attempt larger local models without first buying enough GPU memory to hold everything. It also makes the storage device part of every capacity decision.

Our 30.9 MB checkout is easy to mistake for the size of the job. It contained 1,191 files and roughly 342,274 source lines, before any model download. The README separates models that fit in memory from larger ones that stream experts. That distinction matters more than the size of the launcher: a compact program can still depend on a large weight collection and spend its time waiting for reads.

What happened when we ran it

Our run installed 36 Python packages in 28 seconds and built successfully in 3 seconds at commit bf24429. Installed packages occupied 38 MB. We cloned the project on October 13, 2026, into an unprivileged Debian container with 3 CPUs, 8 GB of RAM, no secrets, and the uv Python 3.12 Bookworm image. Pip-audit reported 0 known vulnerabilities in the environment it inspected.

The harness skipped tests because it found no test script or target. Its scan recorded 4 CI workflow files, no Dockerfile, and no tests directory. Those observations describe this checkout and the path our harness recognized. They do not establish that upstream has no checks elsewhere. More importantly, the successful package build gives us no measured answer about generation quality, response speed, or whether a particular model fits your machine.

The setup wizard chooses hardware paths for you

The README's local route begins with platform packages and a setup script. On Debian, it names Git, Python, and compiler tools. The wizard examines hardware, recommends a model, obtains an engine, downloads weights with resume support, and opens a browser dashboard. That sequence is more useful than asking a newcomer to select cache parameters before they have seen a reply. It also leaves meaningful choices available through explicit backend and model options.

The 38 MB installed footprint excludes the weights that make the application useful. Disk location deserves attention before you accept a download recommendation. The documentation warns WSL users to keep models on the Linux filesystem rather than under /mnt/c. CUDA depends on the applicable toolkit and engine, while Vulkan needs its driver and tooling. On macOS, automatic setup builds for CPU, so using the documented Metal backend calls for another deliberate choice.

API compatibility depends on the selected engine

Colibri exposes OpenAI-compatible chat and completion routes alongside an Anthropic-compatible messages route. The browser interface shows chat and expert activity, and an MCP server can inspect hardware or manage the local installation. This is a practical fit for someone who already has a coding client and wants to point it at local inference. You can retain the familiar client while investigating how the engine places and reads weights.

A 3-second build cannot certify all those interfaces. The README explicitly excludes Inkling and OLMoE from tool calling, and image input is available only on named model families. Treat the per-engine table as part of application design. If your agent requires tools, choosing an unsupported engine breaks the intended workflow even when ordinary chat works. Check the features your client actually sends rather than relying on the broad compatibility label.

An Inkling report matches our measured commit

Issue #2132 describes Inkling exiting during prefill at commit bf24429, the same revision as our sandbox checkout. The reporter includes a cache error saying the engine served a different expert from the one requested, on a Vulkan configuration. This is a user report about that engine and setup, not a crash we reproduced. It is still a specific reason to verify Inkling before depending on it for long unattended prompts.

GitHub showed 42,453 stars, 128 open issues and PRs, and an October 17, 2026 push. Release v2.0.0 was published on October 6. Recent activity includes hardware reports and proposed fixes for setup detection and engine behavior. Taken together, those signals show active development rather than abandonment. They also suggest that updating can change relevant behavior, so keep the model, engine revision, and backend together in any reproducible deployment record.

Start with the exact workload you need to keep

Colibri earns a trial when learning how a large model behaves on your existing machine is itself useful. Its routing dashboard and explicit storage choices support that work. If you mainly want routine local chat, Ollama is an easier starting point to evaluate. If a supported GGUF model already fits your workflow, compare llama.cpp before taking on Colibri's engine-specific decisions. Neither comparison requires accepting an unmeasured speed claim.

The 28-second install lowers the cost of examining the repository, but adoption rests on the next task: run your own prompts through the chosen model, backend, and client. Include tool calls if the application needs them, and watch what happens after the cache changes. Keep Colibri if that exact setup does useful work at a pace you can tolerate. A successful package check alone is too little evidence to make that decision.

Alternatives

ProjectWhat it isPick it when
llama.cpp gh↗A local inference engine with a broad model and hardware ecosystem.pick this instead when your model already has a supported GGUF route and established tooling matters most.
Ollama gh↗A local model runner with a model management CLI and HTTP API.pick this instead when routine local chat and model management matter more than expert placement experiments.
vLLM gh↗A model serving engine aimed at shared inference workloads.pick this instead when your hardware can hold the chosen model and serving concurrent users is the priority.

Sources

  1. Project README
  2. Inkling crash report at bf24429
  3. v2.0.0 release
  4. Repository activity

More llm tools reviews