EN
Book a call

Local AI · Recipe · 2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context): 89.3 tok/s according to github.com (dataset as of Sep 29, 2026).

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

Install: Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

Command
git clone https://github.com/MiaAI-Lab/Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold.git && cd Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold && cp scripts/local.sh.example scripts/local.sh && ./start.sh

Full instructions in the README Repo

Engine

Engine
TensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)
Quantization
INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KV
Model family
Qwen3.8-Flash-Next
Context
1,048,576
Parameters
125B MoE / 4,8B aktiv (Top-5-Experten)
Creator
MiaAI-Lab
GitHub stars
15
Repo updated
Oct 8, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)89.3 tok/scontext 1024K · Prosa, sparkDash 2026-10-07, QUANT=int4ar; TTFT 76 ms Source 

Decode · parallel requests

Sums over all streams are marked; per request it is correspondingly less.

  • 16 parallel requests484 tok/sPeak (decode) · context 1024K · Prosa ×16 aggregiert (35,1 pro Request); ×8 284,9 Source 

Prefill by context

  • 8K3,303 tok/sFrüher am selben Tag gemessen, bf16 KV, PARALLEL 8; 4k 3.244 / 32k 3.285 / 128k 2.982 / 256k 2.673 tok/s Source 

Time to first token

Shorter is better.

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
TensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required
  • vision
  • tool-calling
  • License unclear

Sources

Related recipes

2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 6, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B NVFP4 with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

67.4tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4 (nvidia, Experten) + FP8 n-gram/MTP, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-DraftsWeights Repo Updated Oct 8, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview