Edge0 reached 580 GitHub stars in MrKeyoor's 12:20 UTC snapshot on September 10, less than 47 hours after GitHub recorded the repository's creation. The burst of interest follows a sharp systems claim: its 35B mixture-of-experts model generated 14.9 to 17.7 tokens per second on a 24GB M4 Pro Mac mini while reporting 2.9 GiB of peak active memory. If other developers can reproduce that result, a Mac with ordinary memory could run a class of model usually associated with much larger memory budgets.
The 2.9 GiB figure needs its full label. It is the project's measurement of the MLX allocator at short context, rather than the size of the model or the Mac's total memory use. The 35B model card lists 19.6GB of model files, and Edge0 keeps those weights on storage. Its documentation also says warm runs benefit from the operating system's page cache. Edge0 has found a way to limit the weights active in MLX; it has not compressed 35 billion parameters into a 2.9 GiB file.
That distinction makes the release more interesting, because the Apache-2.0 repository contains a specific implementation rather than a memory-saving slogan. Edge0 combines 4-bit weights, selective expert loading, a cache, and an extra network that predicts which experts the model will need next. The published results all come from the Edge0 team. There is no independent performance reproduction in the material reviewed for this article.
Sparse models create the opening
Edge0 ships two preview bundles. The larger one starts with Qwen3.5-MoE 35B-A3B and has 40 layers containing 256 experts, of which four are active for each token. The smaller bundle is based on inclusionAI's Ling 3.0 tiny model, with about 7.9 billion total parameters and 1.2 billion active parameters. It has 128 experts and selects eight per token, according to the 8B model card.
A mixture-of-experts model gives Edge0 a useful property: most expert weights are idle for a given token. The normal router chooses a small subset, so loading every expert into fast memory wastes capacity. Edge0's SSD streaming design says the quantized expert weights occupy about 310MB in each of the 35B model's 40 layers. Instead of keeping all of them resident, the runtime maps Safetensors files and reads the byte ranges for the chosen experts.
The runtime maintains a shared least-recently-used cache and fixed slots for the experts expected on the next step. A fast staged path performs the usual matrix operations against those slots. If a needed slot is not ready, the code falls back to an exact path that fetches the expert on demand. During prompt processing, the 35B preset can load whole expert layers for the first 12 layers, then free each GPU copy after use. These are concrete tradeoffs between memory and storage traffic, exposed in code and configuration.
Apple's MLX is a good fit for the experiment. Its unified-memory documentation explains that CPU and GPU operations share one memory pool on Apple silicon, without moving arrays between separate host and device stores. Edge0 builds its current backend around MLX and memory mapping. Fast internal storage and the macOS page cache become part of the inference path.
The router has to outrun the disk
Edge0's prerouter notes describe an awkward dependency in ordinary MoE routing. A layer must produce its output before the next layer's router can choose experts. Waiting for that decision and then reading weights from SSD would insert storage latency into every decode step. Edge0 adds a small network that tries to make the decision early.
The prerouter design uses a layer's hidden state and recent routing choices to predict the following layer's expert selection one token ahead. The prediction lets the loader fill the fixed slots while generation continues. The project reports as much as a 59% decode-throughput improvement, with the gain varying by storage latency, model size, and the number of selected experts. That maximum is a project result, not a general speedup for every Mac or prompt.
Prediction also changes the path through the model, while 4-bit quantization reduces numerical precision. Edge0 addresses the quality loss with what it calls Recover-LoRA. The base weights stay frozen, and LoRA adapters are trained by distillation from the full-precision model. Each downloadable directory includes the base checkpoint, LoRA weights, and prerouter weights. The runtime loads them together, so the preview is a coupled model-and-inference package rather than a drop-in engine for any MoE checkpoint.
The team's five-test OpenCompass table reports an average score of 79.2 for Edge0-35B, compared with 83.2 for the full-precision Qwen base. HumanEval falls from 95.1 to 90.9, while AIME 2026 falls from 92.7 to 86.6. The 8B package averages 69.9 against 72.7 for its base. Its MMLU-Pro result rises from 65.8 to 70.1, but AIME drops ten points and IFBench drops 6.7. Averages hide those task-level differences, which matter more than a single claim that most quality was recovered.
The benchmark has a narrow frame
Edge0's performance table comes from two runs per model on one 24GB M4 Pro Mac mini. Its benchmark uses a prompt of about 3,300 tokens, ten sampled warmup steps, and 200 timed decode tokens. The 35B tier reports 14.9 to 17.7 tokens per second during decode, plus prefill rates of 113 tokens per second from a cold start and 140 when warm.
The 8B tier reports 23.9 to 25.3 tokens per second. Its prefill rate moves from 500 tokens per second cold to 1,428 warm. The project's benchmark notes attribute the warm result to cached model data. The 8B model card applies the 1.0 GiB peak to short contexts, while a roughly 3,300-token context raises the figure to about 3.3 GiB because of the key-value cache. Readers should avoid comparing the headline memory number with a long-context server allocation as though they measure the same run.
The repository includes its benchmark script, unit tests that do not require model weights, and slower tests for real generation. That is enough for an Apple-silicon owner to check the claims on another machine. A useful reproduction would record total process memory, page-cache behavior, bytes read from storage, first-token latency, sustained decode speed, and power use. The project's current table reports only part of that operating profile.
Preview means Mac-only and model-specific
Despite the model cards' phrase "phone-class memory," Edge0 does not currently support phones. The repository's requirements specify macOS on Apple silicon and name M1 through M4 systems. Python 3.10 or newer is also required. A CUDA backend appears on the roadmap, with an empty backend slot reserved in the architecture, but the public release supports only MLX.
The server side is deliberately familiar. Once the model directory is downloaded, the documented command edge0 serve edge0-35b starts an OpenAI-compatible /v1/chat/completions endpoint. A minimal local request looks like this:
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Hello"}],"max_tokens":32}'
Compatibility at the HTTP boundary should make experiments easy, though the 35B card says this preview is weak at tool use, multi-step planning, and long-running agent work. Its low active-memory number also depends on short contexts. Developers evaluating it for a coding agent need to test the agent workload and context length they plan to use, rather than carry over the chat benchmark.
Edge0's quick rise is understandable. The streaming code and documentation turn sparse routing into a storage-scheduling problem and give others enough detail to challenge the result. The next evidence to watch is a reproduction across several Mac generations and SSD conditions, with total system memory and long-context behavior reported alongside MLX allocation. CUDA support would test whether the backend boundary is real. Until those measurements arrive, 2.9 GiB is a well-defined project result from one Mac, not a general hardware requirement for a 35B model.