Local AI · Recipe · 2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.0 bpw with vLLM on 2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.0 bpw with vLLM on 2× DGX Spark: 42.6 tok/s according to github.com (dataset as of Nov 3, 2026).

by sfxnz

42.6tok/sEveryday

1 request, prose prompt, with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis40with thinkingno thinking 25Original model's value. Heavily compressed (~2 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality

Engine

Engine
vLLM
Quantization
EXL3 2.0 bpw MCG (routed experts, tail-biting Viterbi re-encoded, scale refit), lm_head MXFP8, KV fp8 (8-GiB-Pin), DSpark-3
Model family
DeepSeek-V4.1-Flash
Context
1,048,576
Parameters
Basis deepseek-ai/DeepSeek-V4.1-Flash: 763,205,315,794 Parameter (HF-API safetensors total); EXL3-2.0bpw-Pack ~334 GB auf der Platte
Creator
sfxnz
GitHub stars
23
Repo updated
Sep 27, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)42.6 tok/scontext 512-token prose · L.A.I.L-Zelle (tools/measure_lail_prose.py), n=20 gepoolt über 2 Boots (V-1/V-2), Round 36; ms/step 48.78; +85 % vs. published 23 Source 
  • Peak (decode)84.2 tok/sBench_decode.py 'structured' (Count 1→200), Median; Round 36 arm_median 84.279 Source 

Prefill by context

  • 8K838 tok/sRound 34 (Review-Kampagne, Engram WILLNEED read-ahead); s13-Boot Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Experimental
  • Custom kernel required

Sources

Related recipes

8× DGX Spark
74tok/sEveryday

1 request, generation at 32K context (speed table, ring of eight), thinking on, high, with DSpark (per recipe); experimental profile Source 

  • Custom kernel required
  • Experimental

Stock-Checkpoint FP8/MXFP4 (DeepSeek)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.