NVIDIA releases Nemotron 3.5 Bolt, a 74B open model packed into 24GB for local agents

What NVIDIA released

NVIDIA published Nemotron 3.5 Bolt 74B-A9B 24GB on 15 October 2026. The technical blog describes a mixture-of-experts model with a hybrid Mamba and Transformer design, built to run everyday agentic tasks locally on RTX and DGX Spark machines. The model card lists 74 billion total parameters, 9 billion active parameters, a context of up to 256K tokens, and text, image and video input with text output.

The model was compressed from Nemotron 3.5 Super VL, a 120B-A12B model, using an NVIDIA method called Star Elastic. NVIDIA then quantized the expert weights to IQ2_XXS at 2.0625 bits per element and recovered accuracy with quantization-aware distillation. Mamba layers use NVFP4 and the embedding, attention and output layers use Q8_0. The release includes a GGUF build for llama.cpp and a 1.13B-parameter LiLiCorr drafter for speculative decoding. The license is OpenMDW 1.1, and the model card says the model is ready for commercial and non-commercial use.

Benchmarks NVIDIA reports

NVIDIA compares Bolt with open models that fit in 24GB at 4-bit precision. On its model card, Bolt scores 86.43 on GPQA Diamond against 83.40 for Qwen3.6-35B-A3B and 79.61 for Gemma 4 26B-A4B. It scores 50.98 on Terminal-Bench 2.1 and 66.75 on BrowseComp, where Qwen3.6-35B-A3B scores 44.38 and 48.74. Bolt trails on two rows: AA-Omniscience accuracy, where Gemma 4 leads, and PinchBench, where Qwen leads by a fraction of a point.

NVIDIA says the drafter lifts decoding to 84 tokens per second on the AIPerf SPEED-Bench benchmark on RTX Spark. These are vendor figures. Independent runs on consumer hardware were not part of the announcement.

Why it matters for open models

Most local models are trained small and quantized to 4 bits. NVIDIA argues the opposite route works better under a fixed memory budget: start from a large model, prune it, and push the weights down to 2 bits. If outside tests confirm the numbers, developers with a 24GB GPU get a multimodal agent model with a long context that does not need a cloud API. NVIDIA's blog post on RTX Spark says the model runs through llama.cpp, vLLM, Ollama, Unsloth and LM Studio, and inside agent apps such as Hermes and OpenClaw.

The release adds to a busy month for open weights, following Aleph Alpha's Kolibri-1. More releases are collected under Open Source AI and on the model release tracker.

What to watch next

The vLLM recipe on the model card lists NVIDIA Ampere, Hopper and Blackwell data center GPUs, and the card names the GeForce RTX 5090, DGX Spark and RTX Spark as tested hardware. Support for older 24GB consumer cards is not stated in the announcement. NVIDIA also says a smaller Nemotron 3.5 Bolt E4B 4GB model for game characters arrives on 23 October. The next check is how the 2-bit model holds up against 4-bit rivals in community benchmarks.

Questions people ask

How much memory does Nemotron 3.5 Bolt need?

NVIDIA says the packed model fits in 24GB of memory. Most expert weights are stored at about 2 bits per parameter.

Can Nemotron 3.5 Bolt be used commercially?

Yes. The model card says it is ready for commercial and non-commercial use under the OpenMDW License Agreement 1.1.

Which tools can run Nemotron 3.5 Bolt?

NVIDIA lists llama.cpp, vLLM, Ollama, Unsloth and LM Studio, and publishes a GGUF build on Hugging Face.

Sources

1. NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (NVIDIA Research, 15 Oct 2026)
2. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB model card (NVIDIA on Hugging Face, 15 Oct 2026)
3. All Systems Go: Global PC App Ecosystem Ready for NVIDIA RTX Spark (NVIDIA Blog, 13 Oct 2026)