A 48.4GB language-model file can run on a laptop with 32GB of RAM if the machine keeps the right parts close at hand. Janas's model catalog describes an experimental inference engine that reads selected experts from an NVMe SSD and caches the ones it needs repeatedly. For someone choosing a local model, that changes the first question from whether the entire download fits in memory to how much work each token sends to disk.
Author Maurizio Cammalleri reports 24 tokens per second from Qwen3-Next without speculative drafts, against 2.8 for a specially configured llama.cpp. MrKeyoor has not reproduced those measurements. The memory design deserves attention before the speed ratio becomes a buying guide.
Why an 80B model doesn't need every expert at once
Qwen's official model card lists 80 billion total parameters and 3 billion activated parameters. Its mixture-of-experts layers have 512 experts, with 10 activated and one shared expert. A router selects which specialized feed-forward networks process each token. That sparse computation gives an inference engine a smaller immediate working set than the full collection of weights stored on disk.
The distinction matters because a model's file size and the weights it uses for one token answer different questions. Qwen's 80B total describes the available capacity. Its 3B activated figure describes the parameters involved in a token's computation. You still need access to the other experts when the router selects them. The model architecture makes selective loading possible without suggesting that the entire model has shrunk to 3B.
Janas reorganizes GGUF weights into its own .jns format. According to its conversion documentation, it creates one slot per layer and expert and places experts by co-activation so reads fall near one another. Conversion itself doesn't retrain or prune the model. The point is to let the engine fetch the weights a token calls for without reading unrelated experts alongside them.
That format also supports splitting an expert's down matrix into three two-bit planes. Reading all three preserves the original weights, the same document explains. Reading fewer planes gives the engine another way to reduce the bytes it moves. File conversion and the precision selected at runtime are separate decisions, so an unchanged model file doesn't by itself establish identical computation under every setting.
What the laptop comparison establishes
The comparison used Debian 13, an Intel Core Ultra 9 185H with integrated Arc graphics, 32GB of DDR5 and an NVMe SSD reading at about 5.7GB/s. Janas used roughly 19GB of expert cache. It reports 62 tokens per second reading a 1,024-token prompt, versus 16 for llama.cpp.
The author says llama.cpp's usual build ran out of memory with GPU participation. Running the model required disabling CPU weight repacking and leaving experts in the memory-mapped file. The methodology describes single rounds for this model, with llama-bench generating without a preceding prompt and janas-bench generating after one. Those differences limit what the comparison establishes about another computer or a longer workload.
A faster answer can mean two different things
Janas's chat documentation describes speculative decoding: a prediction block guesses upcoming tokens, then the main model checks them. At the first disagreement, the guess is discarded and the main model's token is used. The project says tests with the same seed produce identical text with and without drafts. This mechanism batches useful guesses into fewer passes without deliberately accepting a weaker answer.
The gains depend on the text and available cache. The chat notes report little benefit for a small Qwen model on Italian prose and no gain when Qwen3-Next's cache is so small that checking several tokens adds too many disk reads. A drafting result measured on one explanation shouldn't become an expectation for your source code or language.
Other settings do change computation. The feature documentation says Janas can skip lightly weighted experts that aren't cached when the cache holds less than one-fifth of the experts. It redistributes their weight to the token's remaining experts. That buys speed by doing less work, and the document records changes in agreement with reference outputs.
A practical reading of those accuracy checks is to separate generating the same text faster from accepting numerical or routing changes. Matching the most likely token on a short reference passage doesn't measure whether an omitted expert affects a difficult debugging task. If you test a small-cache configuration, judge the answers on your work as well as the reported token rate.
The download still needs somewhere to live
The model preparation guide lists the Qwen3-Next GGUF and converted file at the same 48.4GB. Manual conversion with the optional bit-plane step can temporarily require three copies, about 145GB for the largest model. Preconverted downloads avoid that local conversion workflow. Running beyond RAM capacity therefore leaves a substantial storage requirement, even before you consider other models.
The installation requirements call for an NVMe SSD and recommend 32GB or more for large models. Qwen3-Next has about 2GB of resident weights, with the expert cache taking additional memory. A Vulkan 1.3 GPU is optional. The engine measures whether it helps, and it can run entirely without one. Hardware suitability depends on the path from storage to computation, alongside available RAM.
For a machine you already own, the documented benchmark command is more useful than extrapolating the development laptop's result:
bin/x86_64-linux/janas-bench qwen3-next-80b-a3b
It measures configurations and writes a report. The tool can offer to download a missing catalog model, so the command is a substantial test rather than a small diagnostic. The installer's tuning notes also describe cache controls and ongoing thread selection. Leave room for other applications when testing a laptop you still need to use.
Putting the model behind an existing client
Janas includes a local HTTP server, making the memory experiment usable beyond a terminal. Its serving documentation describes an API following OpenAI's request formats at http://127.0.0.1:8080/v1, with chat completions and streamed replies. It also provides janas-mcp, which exposes generation and embeddings as tools to MCP clients. These are documented interfaces, rather than a requirement to build a new application around the C library.
The server keeps computed conversation state so a client sending the same history plus another message can avoid rereading unchanged material. The serving notes say changes earlier in a conversation cause work to be repeated, including a full reread for models with recurrent state such as Qwen3-Next. That connects directly to the SSD proposition: tolerable generation speed doesn't guarantee a short wait after a large or changed prompt.
The next useful evidence comes from other machines
An October 12 changelog entry adds a Windows x86-64 build and reports tests on Windows 11 using the same development laptop. This broadens who can try the project, while leaving the hardware comparison narrow. The platform addition is timely, but the larger unanswered question is whether selective expert loading pays on the machines developers actually have.
The hardware reporting guide asks for tests on AMD processors, CPUs without AVX-VNNI, smaller-memory machines and slower disks. It records an RTX 5080 user's result, while still identifying many GPU configurations as unknown. It also warns that a GPU driver reset can take down the desktop. These are concrete gaps for anyone considering Janas as a daily inference engine.
Watch for reports that pair the exact model and cache settings with disk type, prompt length and answer checks. Janas's reporting instructions already ask contributors for model size, hardware and failures as well as speed. A reproducible result on a slower SSD would tell a developer more about running a 48.4GB model on 32GB RAM than another best-case token rate from the original laptop.