EN
Book a call

Local AI · Recipe · 1× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 1.62 bpw with vLLM on 1× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 1.62 bpw with vLLM on 1× DGX Spark: ~20.1 tok/s according to github.com (dataset as of Nov 3, 2026).

by vcruz305

~20.1tok/sEverydayapproximate · method not stated

1 request, chat prompt (1200-token replies), sampling 0.6/0.95, with DSpark (per recipe), range 17.7–22.5 tok/s

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
vLLM (spark-ds41f r6w, Basis-Image dsv41-1017-ds41f:r6w) + vllm-exl3 Fork 30e8b94 + exllamav3 Fork 909ad85 (tile-K 16, W2) + Shim, DSpark k=5
Quantization
EXL3 SAGE MixedK 1.62bpw (routed K1–K4), FP8 block-32 sonst, EXL3-K8-Head, Engram von NVMe, Cold-Expert-Alias-Datei
Model family
DeepSeek-V4.1-Flash-1017
Context
524,288
Creator
vcruz305
GitHub stars
38
Repo updated
Oct 28, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)20.1 tok/sChat-Antworten (1200 Token, 3 Prompts), Sampling-Default 0,6/0,95, DSpark k=5; Bereich 17,7–22,5 tok/s Source 
  • Mixed (decode)35.1 tok/sBench7: je 3 Prompts Prosa/Code/strukturiert, 512 Token, greedy, W2-Kernel, max-model-len 524288, Mittel aus 4 Boots Source 

Prefill by context

  • 14K720 tok/sVaried-Text-Leiter (stdlib-Quelltext, ca. 14.000 Token), Chunk 2112; README nennt ~720 tok/s als Realtext-Wert Source 

What you need

Hardware
1 × NVIDIA DGX Spark (GB10)
Engine
vLLM (spark-ds41f r6w, Basis-Image dsv41-1017-ds41f:r6w) + vllm-exl3 Fork 30e8b94 + exllamav3 Fork 909ad85 (tile-K 16, W2) + Shim, DSpark k=5
Context
524,288 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • tool-calling

Sources

Related recipes

4× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 4.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

70.9tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 4.0 bpw (4,02 bpw, ~275 GB; eigener Quant)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @0a0b69a, recipe v0.3.5) + kernel kit (362 files), TP4 über RoCE, DSpark drafts, CUDA graphsWeights Repo Updated Oct 24, 2026

Details
4× DGX Spark

DeepSeek-V4.1-Flash-1017 FP8/MXFP4 with TensorFold on 4× DGX Spark (switchless ring)

by MiaAI-Lab

65.8tok/sEveryday

1 request, prose prompt (T = 0.7, thinking on), with DSpark (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required

FP8 / MXFP4 (official checkpoint as published, no re-quantization)TensorFold native (Zig 0.17.0), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @5940208, kit 6033ea4) + recipe zig/patches, TP4 over a switchless ConnectX-7 ring (./start-switchless.sh), DSpark drafts, CUDA graphsWeights Repo Updated Nov 2, 2026

Details
2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 EAQ 2.9 bpw with TensorFold on 2× DGX Spark

by MiaAI-Lab

63.9tok/sEveryday

1 request, prose prompt (temperature 0.7, thinking on), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 EAQ v1 2.9 bpw (Mia-AiLab; vision tower and DSpark draft blocks included)TensorFold native (Zig 0.17.0), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig @fbbc971) + recipe patches, TP=2 over ConnectX-7 (RoCE), DSpark draftsWeights Repo Updated Nov 2, 2026

Details
2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.5 bpw with TensorFold on 2× DGX Spark

by bertholomus

60.5tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 2.5 bpw routed experts (2,52 bpw über die quantisierten Gewichte, ~185 GB; Attention/Shared Experts 4 bpw, DSpark-Drafter 4/6 bpw, Vision-Tower 16 bit)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp2, recipe v0.7.3, served engine 6579b756dcd7) + kernel kit v0.7.0, TP=2 über RoCE, DSpark draftsWeights Repo Updated Oct 24, 2026

Details
2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw (LibertAI) with TensorFold on 2× DGX Spark

by bertholomus

59.2tok/sEveryday

1 request, prose prompt (temperature 0, thinking off, single run), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 2.9 bpw (LibertAI, Revision afc7aa0a)TensorFold (Elevatormusic/TensorFold @07aa02585f64: TensorFold 0.6.6 line + BertholomusAI DeepSeek-V4.1 family), Launcher Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark, TP=2 über ConnectX-7, DSpark (MTP_DRAFTS=5), MEM_FLOOR_GIB=0Weights Repo Updated Oct 19, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy