IT
Prenoti una chiamata

Local AI · Recipe · 2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.77 bpw with vLLM on 2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.77 bpw with vLLM on 2× DGX Spark: ~29 tok/s according to github.com (dataset as of Sep 29, 2026).

by 0xSero

~29tok/sEverydayapproximate · method not stated

1 request, prose prompt, with DSpark (7 draft tokens) (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis40with thinkingno thinking 25

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
vLLM-Fork (EXL3-MoE + B12X-Kernel) ghcr.io/0xsero/deepseek-v4.1-flash-spark@sha256:5668e35e
Quantization
EXL3 MUL1 2.77 bpw routed experts (K2/K3/K5-Mix), Rest upstream unverändert; FP8 KV
Model family
DeepSeek-V4.1-Flash
Context
262,144
Creator
0xSero
GitHub stars
90
Repo updated
Oct 3, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)29 tok/sProse decode, 1 stream, „~29“; TP2 über RoCE, natürliches Stop ohne Output-Cap; gemessen mit historischem Image 5668e35e + voller Upstream-Checkpoint mit separaten Expert-Banks (nicht als Ergebnis des kompakten Public-Checkpoints/CI-Image ausgewiesen) Source 

Decode · parallel requests

Sums over all streams are marked; per request it is correspondingly less.

  • sum over 2 streams60 tok/sPeak (decode) · Code decode, 2 streams aggregate; Code 1 stream ~39–41 tok/s Source 

Prefill by context

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM-Fork (EXL3-MoE + B12X-Kernel) ghcr.io/0xsero/deepseek-v4.1-flash-spark@sha256:5668e35e
Context
262,144 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • vision
  • tool-calling

Sources

Related recipes

2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.0 bpw with vLLM on 2× DGX Spark

by sfxnz

42.6tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source 

Intelligence40with thinkingno thinking 25Original model's value. Heavily compressed (~2 bit), intelligence is likely much lower.
  • ≤ 3 bit: quantization may cost quality
  • Experimental
  • Custom kernel required

EXL3 2.0 bpw MCG (routed experts, tail-biting Viterbi re-encoded, scale refit), lm_head MXFP8, KV fp8 (8-GiB-Pin), DSpark-3vLLMWeights Repo Updated Sep 27, 2026

Details
2× DGX Spark

DeepSeek-V4.1-Flash EXL3 2.0 bpw with TensorFold on 2× DGX Spark

by sfxnz

41.2tok/sEveryday

1 request, prose prompt (512 tokens), with DSpark (per recipe) Source 

Intelligence40with thinkingno thinking 25Original model's value. Heavily compressed (~2 bit), intelligence is likely much lower.
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 2.0 bpw MCG (routed experts, Branch 2.0bpw-mcg-viterbi-lmhead-mxfp8 @982b704), Rest FP8 (32x32-Blockscales), lm_head MXFP8, DSpark-Draft MXFP4TensorFold Fork sfxnz/TensorFold dsv41-recipe-engine4 @b86514a (v0.6.4 + deepseek_v41-Familie), TP=2, --parallel 4Weights Repo Updated Oct 8, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview