Nvidia released Nemotron 3.5 Bolt 74B-A9B 24GB on October 15, giving developers another downloadable model for running AI agents locally. The company compressed a larger model and reduced the precision of its weights to target a 24GB footprint. Its launch announcement puts Bolt alongside the larger Nemotron 3.5 Super VL, with a smaller Bolt E4B model scheduled for October 23.
The 24GB name needs a qualification before you plan a deployment. Nvidia's current GGUF model card lists 28GB and 32GB serving recipes, with different hardware and context limits. This article examines the published architecture and configurations. We haven't run Bolt or checked its visual outputs.
Compressing the parent model
The model has 74 billion parameters, with nine billion active per token. Nvidia's research account describes compressing the 120B-parameter Super VL parent before further training to recover capabilities. Its expert weights use IQ2_XXS at 2.0625 bits per element. Other components use higher precision, so the two-bit description doesn't apply to every weight.
The release combines that compression with a separate LiLiCorr token drafter. Speculative decoding lets a smaller network propose tokens for the main model to verify. Nvidia's technical explanation describes LiLiCorr scoring candidates across an entire draft together. That makes the drafter part of the configuration behind a speed result, rather than an optional detail you can omit when reproducing it.
Budget for the serving recipe
Nvidia's llama.cpp deployment table separates the packed model from the configurations used to serve it:
| Documented recipe | Listed hardware | Context limit |
|---|---|---|
| 28GB | RTX Spark 64GB or discrete RTX GPU with 32GB or more | 65,536 tokens |
| 32GB | RTX Spark 128GB or discrete RTX GPU with 48GB or more | 262,144 tokens |
Both recipes allow one concurrent sequence and use a separate speculative drafter. The smaller recipe also specifies an eight-bit key-value cache. Those settings matter if you're filling an agent session with repository files and tool results: the model card says prompts, visual inputs and generated responses share the context window.
These are documented configurations, not measured minimum hardware requirements from MrKeyoor. A developer choosing a machine should start with the matching recipe and verify memory use under the intended workload.
Keep the speed result attached to its machine
Nvidia reports 83.99 generated tokens per second on AIPerf SPEED-Bench-Coding with the IQ2_XXS model and LiLiCorr. Its measurements used llama.cpp on a 128GB Surface Laptop Ultra with RTX Spark hardware. That result doesn't establish the speed of an arbitrary desktop GPU or a complete coding-agent job.
The next useful evidence is a reproducible run on the hardware developers actually intend to use, reporting memory consumption and successful task completion at the chosen context limit. Nvidia's published serving configurations give those tests a starting point. Whether the 28GB recipe leaves enough room for your agent's history still needs checking on that setup.