EN
Book a call

Local AI · Recipe · 1× DGX Spark

Nemotron-3.5-Lightning 30B NVFP4 with SGLang on 1× DGX Spark

Nemotron-3.5-Lightning 30B NVFP4 with SGLang on 1× DGX Spark: 66.7 tok/s according to github.com (dataset as of Nov 3, 2026).

by r0b0tlab

66.7tok/sMixed

1 request, mixed prompt set, with DFlash

Source: github.com
Intelligence (original model) · Artificial Analysis13with thinking
Instructions in the README Repo

Engine

Engine
SGLang
Quantization
NVFP4 (W4A16 Marlin MoE) + FP8 KV
Model family
Nemotron-3.5-Lightning
Context
50,016
Parameters
30B total / 3B active (laut Name)
Creator
r0b0tlab
GitHub stars
0
Repo updated
Aug 21, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Mixed (decode)66.7 tok/scontext 2048 Output (ignore_eos) · Perf-Matrix thinking-off, C1 2048 ignore_eos, DFlash b4, Median; No-spec AR 69.03 Source 
  • Peak (decode)69 tok/scontext 2048 Output (ignore_eos) · No-spec AR, C1 2048 ignore_eos, Median Source 

Prefill by context

  • ~7k Token (cold)4,615 tok/sPrefill proxy cold, DFlash-Lauf Source 

What you need

Hardware
1 × NVIDIA DGX Spark (GB10)
Engine
SGLang
Context
50,016 tokens

Sources

Related recipes

1× DGX Spark

Nemotron-3.5-Bolt 74B-A9B 24GB pack with TensorFold on 1× DGX Spark

by MiaAI-Lab

71.8tok/sEveryday

1 request, prose prompt (sampled, thinking off), with LiLiCorr (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • License unclear

24GB pack: IQ2_XXS experts (2.0625 bits), NVFP4 Mamba projections, Q8_0 attention/latent projections/lm_head; bf16 KVTensorFold tensorfold-native v1.1.6 (eea9dc9c) + recipe patches/ (Bolt lane backend, libbolt.so CUDA forward pass), LiLiCorr draftsWeights Repo Updated Nov 1, 2026

Details
1× DGX Spark

MiniCPM-V-4.7-35B-A3B EXL3 6.08 bpw with ExLlamaV3 on 1× DGX Spark (text, 262k context)

by r0b0tlab

59.4tok/sEveryday

1 request, chat prompt (20 prompts, thinking off), no drafting (default profile) Source 

Intelligenceno independent value
  • Custom kernel required

EXL3 6.08 bpw (28,3 GB), FP16 KV; nur Textpfad (Vision-Tower wird von ExLlamaV3 nicht ausgeführt)ExLlamaV3 1.5.0 (Fork r0b0tlab/exllamav3-r0b0t, Branch minicpm-v4.7-dspark @ 5ddae7d12448), torch 2.14.0+cu130Weights Repo Updated Oct 14, 2026

Details
1× DGX Spark

Qwen3.8-Flash-Next NVFP4 (NVIDIA QAD, FP4-Local) with TensorFold on 1× DGX Spark

by MiaAI-Lab

55.6tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVIDIA FP4-QAD-Export (FP4-Local @b9ce18a6): Routed Experts NVFP4 (Gruppe 16), Attention/Shared Expert MXFP8, alle 512 Experten mit 10 pro Token; n-gram-Tabelle per pread von der SSD; FP8 KVTensorFold tensorfold-native (Zig) v1.1.4 (eab85828) + Patches 0001–0017, Image tensorfold-qwen38:v1.1.4-eab8582 (GHCR v1.1.4-eab8582-4baf16137ecd) auf nvcr.io/nvidia/pytorch:26.07-py3, MTP-DraftsWeights Repo Updated Oct 30, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy