IT
Prenoti una chiamata

Local AI · Recipe · 3× DGX Spark

DeepSeek-V4-Flash 305B FP8 with vLLM on 3× DGX Spark

DeepSeek-V4-Flash 305B FP8 with vLLM on 3× DGX Spark: 41.9 tok/s according to github.com (dataset as of Sep 29, 2026).

by MiaAI-Lab

41.9tok/sEveryday

1 request, prose prompt, with DSpark (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value
Instructions in the README Repo

Engine

Engine
vLLM ghcr.io/anemll/dspark-vllm-gx10:0.1.1
Quantization
Original-Checkpoint (FP8/FP4) , TP3-Head-Pad 64→72
Model family
DeepSeek-V4-Flash
Context
1,048,576
Parameters
~305B MoE
Creator
MiaAI-Lab
GitHub stars
1,427
Repo updated
Sep 26, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)41.9 tok/sWarm, Prosa-Prompts, 512 Output-Tokens, TP=3 16 Slots, 2026-09-02 (TP=2: 38.6) Source 
  • Peak (decode)41.9 tok/sWarm, Prosa-Prompts, 512 Output-Tokens, TP=3 16 Slots, 2026-09-02 (TP=2: 38.6) Source 

What you need

Hardware
3 × NVIDIA DGX Spark (GB10)
Engine
vLLM ghcr.io/anemll/dspark-vllm-gx10:0.1.1
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview