IT
Prenoti una chiamata

Local AI · Recipe · 4× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 4× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 4× DGX Spark: 53.5 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

53.5tok/sEveryday

1 request, prose prompt (temperature 0, single run), with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis34with thinking
Instructions in the README Repo

Engine

Engine
vLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1, Image-ID sha256:3430d661…), DSpark K5, FlashInfer B12X MoE, CUDA Graphs, TP4 über Dual-Rail-RoCE v2, 8 Sequenzen / 16.384 Batched-Tokens
Quantization
offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mla
Model family
DeepSeek-V4-Flash-0731
Context
1,048,576
Parameters
~304B MoE (HF-API safetensors total: 304.180.418.494)
Creator
bertholomus
GitHub stars
2
Repo updated
Aug 13, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)53.5 tok/sProse500 (500-Wörter-Erklärtext, kurzer Prompt), 1 Anfrage, Einzellauf (Publication-Smoke mit scripts/benchmark.py), Temperatur 0, DSpark K5 (Akzeptanz 34,07 %), TTFT 0,18 s; README-Median Single-Stream über 9 Läufe (Prompt-Familie nicht aufgeschlüsselt): 66,08 Source 
  • Peak (decode)103 tok/sCount300 (Zählen 1–300), 1 Anfrage, Einzellauf (Publication-Smoke), Akzeptanz 98,81 %; 20 Dataclasses 94,55; README: Median-Aggregat bei 8 Anfragen 185,06 (10 Runden 176,16–192,00) Source 

Time to first token

Shorter is better.

  • Prompt not stated182 msProse500, warm, kurzer Prompt, 1 Anfrage, Einzellauf (0,1815 s) Source 

What you need

Hardware
4 × NVIDIA DGX Spark (GB10)
Engine
vLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1, Image-ID sha256:3430d661…), DSpark K5, FlashInfer B12X MoE, CUDA Graphs, TP4 über Dual-Rail-RoCE v2, 8 Sequenzen / 16.384 Batched-Tokens
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required
  • tool-calling

Sources

Related recipes

2× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 2× DGX Spark

by bertholomus

40.1tok/sEveryday

1 request, prose prompt (temperature 0), with DSpark (per recipe) Source 

Intelligence34with thinking
  • Custom kernel required

offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mlavLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1), DSpark K5, FlashInfer B12X MoE, CUDA Graphs [1, 2, 4, 8], TP2 über RoCE v2Weights Repo Updated Aug 4, 2026

Details
4× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 4.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

70.9tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 4.0 bpw (4,02 bpw, ~275 GB; eigener Quant)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @0a0b69a, recipe v0.3.5) + kernel kit (362 files), TP4 über RoCE, DSpark drafts, CUDA graphsWeights Repo Updated Oct 24, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Con il vostro consenso utilizziamo Google Analytics (Google, USA) e la nostra misurazione dell’audience per capire come viene utilizzato il nostro sito. Le memorizzazioni strettamente necessarie, come la vostra scelta sui cookie, avvengono senza consenso. Potete modificare la vostra scelta in qualsiasi momento tramite «Impostazioni cookie» nel footer. Informativa sulla privacy · Informativa sui cookie