Bonsai 2 reached 307 Hacker News points in about four hours with a number laptop developers want: 5.9GB for a 27-billion-parameter model. The discussion quickly surfaced the practical catch. Those 5.9GB are the language-model weights, and the files currently require PrismML's fork of llama.cpp. The launch pushes a large local model into ordinary memory budgets while adding a runtime dependency that the headline number does not show.
PrismML introduced Ternary Bonsai 2 27B as a compressed version of Qwen3.8 27B. Most weights take one of three values, -1, 0, or +1, with one FP16 scale shared by each group of 128 weights. The technical report puts the shipped PTQ1_0 format at 1.76 effective bits per weight and 5.93GB, down from 53.8GB for the FP16 language model. That is a 9.1-fold reduction. A small recurrent state path and normalization tensors, 0.0976% of the language model's parameters, remain at higher precision and are included in the calculation.
A smaller file does not cap runtime memory
The 5.93GB figure covers the language model alone. Image input adds a separately packaged vision tower of about 0.63GB in the smaller option. PrismML also ships a PQ2_0 language-model pack at roughly 7.2GB, which uses simpler two-bit slots and can be faster to unpack. The MLX package is 8.49GB because its block format stores a scale and a bias for each group, according to the whitepaper's storage breakdown.
Context consumes memory beyond the weight file. Bonsai 2 supports up to 262,144 tokens, but the official launcher chooses a smaller context from the machine's available RAM unless the user overrides it. PrismML's demo documentation warns that forcing the full training context needs plenty of headroom, since the KV cache, activations, and runtime overhead sit alongside the weights. A 16GB laptop can fit the model with a sensible context. The same laptop should not be assumed to hold all 262K tokens inside a 5.9GB budget.
The default setup makes that distinction visible before inference begins. It downloads the 7.2GB PQ2_0 pack plus the vision projector, for about 7.8GB, rather than the smallest PTQ1_0 file. Open WebUI and the optional code interpreter add several more gigabytes to the installation. PrismML says PQ2_0 is the default because its cheaper unpacking improves prompt processing, while PTQ1_0 is intended for tighter memory limits in the Bonsai demo.
The runtime is part of the model
Bonsai 2 stores its ternary weights in a rotated basis. At inference time, the runtime applies a fixed sign change and a blockwise Walsh-Hadamard transform to activations before matrix multiplication. PrismML wrote custom kernels so CUDA and Metal can operate on the packed weights without expanding them back to FP16. That is how the model saves memory bandwidth, and it is also why an ordinary GGUF runner cannot treat these files like familiar two-bit quantization, as the model card explains.
Stock llama.cpp rejects the new PTQ1_0 and PQ2_0 types. PrismML also warns that a development file labeled Q2_0 may load in an unmodified runtime and emit unusable output because the activation transform is missing. The supported path is the company's pinned llama.cpp fork or its setup scripts. The official quick start reduces that path to a clone, a setup step, and a separate server launch:
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
./setup.sh
./scripts/start_llama_server.sh
That fork requirement belongs in a deployment plan. Runtime updates, container images, and hardware checks must stay aligned with the model format. PrismML's status table lists one upstream llama.cpp pull request for the activation transform's CPU path, and it remained open when the release landed. Until the required work reaches the runtimes a team already maintains, Bonsai 2 carries more integration work than a standard GGUF download, even though its weights and demo are available under Apache 2.0.
The 98.2% claim has two denominators
PrismML's current whitepaper reports an average of 83.9 for Bonsai 2 across 20 benchmarks, versus 85.4 for full-precision Qwen3.8 27B. Dividing those scores produces the advertised 98.2% retention. The suite covers math, coding, instruction following, tool use, knowledge and vision. PrismML ran it with EvalScope and vLLM on H100 GPUs, using the same serving and scoring path for each model, according to its evaluation methodology.
The Hugging Face model card carries a different aggregate. It shows 84.78 for Bonsai 2 and 86.32 for FP16 across 14 benchmarks, which also rounds to 98.2%. Its task mix differs from the 20-test suite in the whitepaper. A developer comparing model charts therefore needs the suite name and per-task table, since the same retention percentage currently refers to two sets of tests in PrismML's release materials.
The per-task results put a hard edge on the phrase "near-lossless." Bonsai 2 scored 52.8 on Terminal-Bench 2.1 against 69.7 for the FP16 baseline. On SWE-bench Verified, it scored 60.8 against 80.6. Those results preserve about three-quarters of baseline performance on two long-horizon software tasks, far below the 98.2% aggregate. The whitepaper treats them separately from its 20-benchmark average, which is the right way to read them.
Other categories land much closer. The company's coding average was 81.58 for Bonsai 2 and 82.17 for FP16, while math came in at 96.57 and 97.06. Vision showed a wider gap, 78.59 against 81.64. Compression did preserve most of the measured capability on many shorter tests. It did not preserve every workload evenly, and an aggregate cannot tell a coding-agent team which side of that split its own repository will resemble, based on PrismML's category results.
These are PrismML's measurements. The report publishes the benchmark names, sampling settings, harness choices, task counts, and per-test scores, which makes the claim easier to inspect. The early Hacker News thread includes user experiments and speed reports, but community runs do not reproduce the 20-benchmark evaluation. A team considering the model for code changes or tool calls still needs a test set drawn from its own work.
Smaller is not always faster
The dense PTQ1_0 pack moves less weight data but spends more computation unpacking ternary values. PrismML found it faster for token generation on Ada-generation GPUs and the 72-watt L4, while PQ2_0 won on H100, A100, Blackwell, and Apple silicon. Prompt processing favored PQ2_0 across the reported machines. The throughput table is a useful warning against choosing the smallest file by default.
On the company's tests, PQ2_0 generated 142.5 tokens per second on an RTX 5090 and 27.7 on an M5 Pro laptop. PTQ1_0 reached 96.7 tokens per second on an RTX 4090, compared with 90.9 for PQ2_0 on the same card. These are batch-one tg128 measurements over 128 generated tokens. They describe steady generation on PrismML's kernels, rather than total latency for loading a prompt, calling tools, or running an agent loop, as the measurement notes make clear.
The M5 Pro result shows why the release drew attention. A 53.8GB FP16 checkpoint cannot fit on that laptop, while a packed 27B model can generate at an interactive rate. The trade is a specialized file format, a forked runtime, and performance that changes with the chosen pack and hardware. PrismML exposes those pieces in its model repository, so developers can evaluate the engineering cost instead of treating 5.9GB as the whole deployment story.
What will settle the deployment question
The next useful evidence will be a mainline runtime path and independent repeats of the long-horizon tests. Teams testing the 262K context should publish total resident memory alongside the model-file size, since that is the number operators will have to provision. Until those measurements arrive, Bonsai 2's 5.9GB result establishes an unusually small weight package with a visible runtime bill. It does not show that every 27B workload now fits inside six gigabytes. PrismML's own roadmap still names faster low-bit kernels, smaller KV caches, and lower rotation overhead as unfinished work.