IT
Prenoti una chiamata

Local AI · Recipe · 2× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 2× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 2× DGX Spark: 40.1 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

40.1tok/sEveryday

1 request, prose prompt (temperature 0), with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis34with thinking
Instructions in the README Repo

Engine

Engine
vLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1), DSpark K5, FlashInfer B12X MoE, CUDA Graphs [1, 2, 4, 8], TP2 über RoCE v2
Quantization
offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mla
Model family
DeepSeek-V4-Flash-0731
Context
1,048,576
Parameters
~304B MoE (HF-API safetensors total: 304.180.418.494)
Creator
bertholomus
GitHub stars
0
Repo updated
Aug 4, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)40.1 tok/sProse500 (500-Wörter-Erklärtext, kurzer Prompt), Median aus 3 warmen Läufen, Temperatur 0, DSpark K5 (Akzeptanz 32,10 %), TTFT 0,235 s; gemessen im initialen Graph-8-Qualifikationsprofil vor der Headroom-Härtung (damals GPU-Util 0,80 / 8.192 Batched-Tokens); empfohlenes Default-Profil: 2 Anfragen je 47,74 tok/s Source 
  • Peak (decode)91.9 tok/sCount300 (Zählen 1–300, gebundene Ausgabe), Median aus 3, DSpark-Akzeptanz 98,81 %; 20 Python-Dataclasses: 84,43; Aggregat bei 2 Anfragen (Default-Profil): 95,48 Source 

Time to first token

Shorter is better.

  • Prompt not stated235 msProse500, warm, kurzer Prompt, 1 Anfrage, Median aus 3 (0,235 s) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1), DSpark K5, FlashInfer B12X MoE, CUDA Graphs [1, 2, 4, 8], TP2 über RoCE v2
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required
  • tool-calling

Sources

Related recipes

4× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 4× DGX Spark

by bertholomus

53.5tok/sEveryday

1 request, prose prompt (temperature 0, single run), with DSpark (per recipe) Source 

Intelligence34with thinking
  • Custom kernel required

offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mlavLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1, Image-ID sha256:3430d661…), DSpark K5, FlashInfer B12X MoE, CUDA Graphs, TP4 über Dual-Rail-RoCE v2, 8 Sequenzen / 16.384 Batched-TokensWeights Repo Updated Aug 13, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

by fujitsupolycom

68tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking on, xhigh, with MTP3 (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B NVFP4 with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

67.4tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4 (nvidia, Experten) + FP8 n-gram/MTP, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-DraftsWeights Repo Updated Oct 26, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Con il vostro consenso utilizziamo Google Analytics (Google, USA) e la nostra misurazione dell’audience per capire come viene utilizzato il nostro sito. Le memorizzazioni strettamente necessarie, come la vostra scelta sui cookie, avvengono senza consenso. Potete modificare la vostra scelta in qualsiasi momento tramite «Impostazioni cookie» nel footer. Informativa sulla privacy · Informativa sui cookie