Local AI · Recipe · 2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.7 bpw (mixed-K) with vLLM on 2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.7 bpw (mixed-K) with vLLM on 2× DGX Spark: 40.5 tok/s according to github.com (dataset as of Nov 3, 2026).

by sfxnz

40.5tok/sEveryday

1 request, prose prompt (L.A.I.L cell), with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

≤ 3 bit: quantization may cost quality

Engine

Engine
vLLM (vllm/vllm-openai:deepseekv41-flash-0909 + vllm-exl3 + Engram-on-disk, custom image dsv41-flash-exl3-sm121:e16-rows), TP=2, DSpark with adaptive verify, DeepSeek-API gateway
Quantization
EXL3 MCG trellis 2.7 bpw routed experts (per-layer mixed K, HF revision 2.7bpw-mcg-hq), other tensors source dtypes, lm_head MXFP8, KV fp8
Model family
DeepSeek-V4.1-Flash-1017
Context
1,048,576
Parameters
~763B (HF-API safetensors total der Basis deepseek-ai/DeepSeek-V4.1-Flash-1017: 763.205.315.794); Pack 405,1 GB
Creator
sfxnz
GitHub stars
6
Repo updated
Oct 30, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)40.5 tok/sL.A.I.L-Prosa-Zelle (tools/measure_lail_prose.py: 512 Tokens, Temperatur 0,2), Median aus 10, Boot FINAL 2026-10-20 mit perf2-Defaults (4 Sessions, 500.000-Token-Fenster), Akzeptanz 2,02; TTFT p50 0,25 s. Zweite Prosa-Zelle (greedy, 200 Tokens, über EOS hinaus erzwungen): 51,3 Source 
  • Peak (decode)104 tok/sBench_decode.py structured, 1 Anfrage, 9 Läufe, Akzeptanz 5,88; Aggregat bei 4 verschiedenen Prompts (structured): 201,1 Source 

Prefill by context

  • 64K2,932 tok/sNeuer Text (novel), 64k-Prompt; 4k 1.830 / 16k 2.236 tok/s Source 

Time to first token

Shorter is better.

  • Prompt not stated200 msKurzer Prompt, Prosa, 1 Anfrage, p50 (bench_decode.py) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM (vllm/vllm-openai:deepseekv41-flash-0909 + vllm-exl3 + Engram-on-disk, custom image dsv41-flash-exl3-sm121:e16-rows), TP=2, DSpark with adaptive verify, DeepSeek-API gateway
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • vision
  • tool-calling

Sources

Related recipes

4× DGX Spark
70.9tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 4.0 bpw (4,02 bpw, ~275 GB; eigener Quant)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @0a0b69a, recipe v0.3.5) + kernel kit (362 files), TP4 über RoCE, DSpark drafts, CUDA graphsWeights Repo Updated Oct 24, 2026

Details
4× DGX Spark
65.8tok/sEveryday

1 request, prose prompt (T = 0.7, thinking on), with DSpark (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required

FP8 / MXFP4 (official checkpoint as published, no re-quantization)TensorFold native (Zig 0.17.0), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @5940208, kit 6033ea4) + recipe zig/patches, TP4 over a switchless ConnectX-7 ring (./start-switchless.sh), DSpark drafts, CUDA graphsWeights Repo Updated Nov 2, 2026

Details
2× DGX Spark
63.9tok/sEveryday

1 request, prose prompt (temperature 0.7, thinking on), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 EAQ v1 2.9 bpw (Mia-AiLab; vision tower and DSpark draft blocks included)TensorFold native (Zig 0.17.0), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig @fbbc971) + recipe patches, TP=2 over ConnectX-7 (RoCE), DSpark draftsWeights Repo Updated Nov 2, 2026

Details
2× DGX Spark
60.5tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 2.5 bpw routed experts (2,52 bpw über die quantisierten Gewichte, ~185 GB; Attention/Shared Experts 4 bpw, DSpark-Drafter 4/6 bpw, Vision-Tower 16 bit)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp2, recipe v0.7.3, served engine 6579b756dcd7) + kernel kit v0.7.0, TP=2 über RoCE, DSpark draftsWeights Repo Updated Oct 24, 2026

Details
2× DGX Spark
59.2tok/sEveryday

1 request, prose prompt (temperature 0, thinking off, single run), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 2.9 bpw (LibertAI, Revision afc7aa0a)TensorFold (Elevatormusic/TensorFold @07aa02585f64: TensorFold 0.6.6 line + BertholomusAI DeepSeek-V4.1 family), Launcher Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark, TP=2 über ConnectX-7, DSpark (MTP_DRAFTS=5), MEM_FLOOR_GIB=0Weights Repo Updated Oct 19, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.