Local AI · Recipe · 3× DGX Spark

DeepSeek-V4-Flash 305B FP8 with vLLM on 3× DGX Spark

DeepSeek-V4-Flash 305B FP8 with vLLM on 3× DGX Spark: 41.9 tok/s according to github.com (dataset as of Nov 3, 2026).

by MiaAI-Lab

41.9tok/sEveryday

1 request, prose prompt, with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

Engine

Engine
vLLM ghcr.io/anemll/dspark-vllm-gx10:0.1.1
Quantization
Original-Checkpoint (FP8/FP4) , TP3-Head-Pad 64→72
Model family
DeepSeek-V4-Flash
Context
1,048,576
Parameters
~305B MoE
Creator
MiaAI-Lab
GitHub stars
1,427
Repo updated
Oct 4, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)41.9 tok/sWarm, Prosa-Prompts, 512 Output-Tokens, TP=3 16 Slots, 2026-09-02 (TP=2: 38.6) Source 
  • Peak (decode)41.9 tok/scontext 1024K · TP=3: Decode ≈ +4–13 % pro Stream vs. TP2; ≈200 tok/s aggregiert bei 16 Streams; Prefill 4–13 % langsamer bis 64K, ≈22 % bei 128–256K Source 

What you need

Hardware
3 × NVIDIA DGX Spark (GB10)
Engine
vLLM ghcr.io/anemll/dspark-vllm-gx10:0.1.1
Context
1,048,576 tokens

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.