FR
Réserver un appel

Local AI · Recipe · 1× DGX Spark

Ling-3.0-flash NVFP4 with vLLM on 1× DGX Spark

Ling-3.0-flash NVFP4 with vLLM on 1× DGX Spark: 21.4 tok/s according to github.com (dataset as of Nov 3, 2026).

by r0b0tlab

21.4tok/sMixed

1 request, mixed prompt set, with MTP (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis25with thinking
Instructions in the README Repo

Engine

Engine
vLLM vllm-ling-v3 @ 92c104112
Quantization
NVFP4 (ModelOpt, nur Experten) + FP8 KV
Model family
Ling-3.0-flash
Context
24,576
Creator
r0b0tlab
GitHub stars
1
Repo updated
Aug 8, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Mixed (decode)21.4 tok/sBase AR, CUDA graphs + VLLM_COMPILE, C1 Source 
  • Peak (decode)33.5 tok/sMTP scale 1 + CUDA graphs, C1; 73.2 % acceptance; 32.8 in der vollen Suite; Prompt nicht angegeben, thinking off Source 

What you need

Hardware
1 × NVIDIA DGX Spark (GB10)
Engine
vLLM vllm-ling-v3 @ 92c104112
Context
24,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required

Sources

Related recipes

1× DGX Spark

Nemotron-3.5-Bolt 74B-A9B 24GB pack with TensorFold on 1× DGX Spark

by MiaAI-Lab

71.8tok/sEveryday

1 request, prose prompt (sampled, thinking off), with LiLiCorr (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • License unclear

24GB pack: IQ2_XXS experts (2.0625 bits), NVFP4 Mamba projections, Q8_0 attention/latent projections/lm_head; bf16 KVTensorFold tensorfold-native v1.1.6 (eea9dc9c) + recipe patches/ (Bolt lane backend, libbolt.so CUDA forward pass), LiLiCorr draftsWeights Repo Updated Nov 1, 2026

Details
1× DGX Spark

MiniCPM-V-4.7-35B-A3B EXL3 6.08 bpw with ExLlamaV3 on 1× DGX Spark (text, 262k context)

by r0b0tlab

59.4tok/sEveryday

1 request, chat prompt (20 prompts, thinking off), no drafting (default profile) Source 

Intelligenceno independent value
  • Custom kernel required

EXL3 6.08 bpw (28,3 GB), FP16 KV; nur Textpfad (Vision-Tower wird von ExLlamaV3 nicht ausgeführt)ExLlamaV3 1.5.0 (Fork r0b0tlab/exllamav3-r0b0t, Branch minicpm-v4.7-dspark @ 5ddae7d12448), torch 2.14.0+cu130Weights Repo Updated Oct 14, 2026

Details
1× DGX Spark

Qwen3.8-Flash-Next NVFP4 (NVIDIA QAD, FP4-Local) with TensorFold on 1× DGX Spark

by MiaAI-Lab

55.6tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVIDIA FP4-QAD-Export (FP4-Local @b9ce18a6): Routed Experts NVFP4 (Gruppe 16), Attention/Shared Expert MXFP8, alle 512 Experten mit 10 pro Token; n-gram-Tabelle per pread von der SSD; FP8 KVTensorFold tensorfold-native (Zig) v1.1.4 (eab85828) + Patches 0001–0017, Image tensorfold-qwen38:v1.1.4-eab8582 (GHCR v1.1.4-eab8582-4baf16137ecd) auf nvcr.io/nvidia/pytorch:26.07-py3, MTP-DraftsWeights Repo Updated Oct 30, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Avec votre consentement, nous utilisons Google Analytics (Google, États-Unis) et notre propre mesure d’audience pour comprendre l’utilisation de notre site. Les stockages strictement nécessaires, comme votre choix de cookies, se font sans consentement. Vous pouvez modifier votre choix à tout moment via « Paramètres des cookies » en pied de page. Politique de confidentialité · Politique relative aux cookies