Local AI · Recipe · 3× DGX Spark

DeepSeek-V4.1-Flash 763B EXL3 3.5 bpw with vLLM on 3× DGX Spark

DeepSeek-V4.1-Flash 763B EXL3 3.5 bpw with vLLM on 3× DGX Spark: 29.6 tok/s according to github.com (dataset as of Nov 3, 2026).

by tonyd2wild

29.6tok/sEveryday

1 request, prose prompt, with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis40with thinkingno thinking 25

Engine

Engine
vLLM
Quantization
EXL3 3.5 bpw routed experts (Pollard) + FP8 Rest
Model family
DeepSeek-V4.1-Flash
Context
300,000
Parameters
~763B MoE
Creator
tonyd2wild
GitHub stars
154
Repo updated
Oct 26, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)29.6 tok/sProse, C1 per-stream, exl3tp3a11, Prompt-Set v1, T=0, thinking off Source 
  • Mixed (decode)46 tok/sC1 aggregate über 8 Kategorien (Counting ausgenommen); per-stream Mittel 51.46 Source 
  • Peak (decode)70.1 tok/sKategorie coding, exl3tp3a11 Source 

Prefill by context

  • 93.3351,199 tok/sCold, unique prefix, TTFT 77.865 s Source 

What you need

Hardware
3 × NVIDIA DGX Spark (GB10)
Engine
vLLM
Context
300,000 tokens

Related recipes

8× DGX Spark
74tok/sEveryday

1 request, generation at 32K context (speed table, ring of eight), thinking on, high, with DSpark (per recipe); experimental profile Source 

  • Custom kernel required
  • Experimental

Stock-Checkpoint FP8/MXFP4 (DeepSeek)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.