IT
Prenoti una chiamata

Local AI · Recipe · 2× DGX Spark

Ling-3.1-Flash-1019 EXL3 3.03 bpw with TensorFold on 2× DGX Spark

Ling-3.1-Flash-1019 EXL3 3.03 bpw with TensorFold on 2× DGX Spark: ~38.2 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

~38.2tok/sPeakapproximate · method not stated

Best value reported by source, 1 request, prompt type not stated, 2,222 prompt tokens, range 37.7–38.7 tok/s, with MTP (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value
Instructions in the README Repo

Engine

Engine
TensorFold (bertholomus fork, branch ling-3.1-tp2 @ 72f16e48, bailing_v3 family on TensorFold v0.6.6 CUDA engine), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over direct RoCE, MTP drafting (adaptive depth)
Quantization
EXL3 3.03 bpw (routed experts 3, shared experts 5, KDA attention 4, dense MLP 4, MLA attention 6, head 6, MTP layer 4)
Model family
Ling-3.1-Flash-1019
Context
262,144
Parameters
560B MoE / 25B aktiv
Creator
bertholomus
GitHub stars
2
Repo updated
Oct 29, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Peak (decode)38.2 tok/scontext 2.222 · Decode drafted (MTP), 3 Läufe 37.7-38.7 tok/s (Mittelwert des Bereichs), 2,222 Prompt-Tokens, 60-88 Reply-Tokens, greedy; Prompt-Typ in der Quelle nicht genannt; 72-Token-Prompt (thinking off) 35.1-36.9, 6,623 Tokens 36.9-39.3, 255,112 Tokens 28.8-29.1, serial 22.9-23.0 Source 

Prefill by context

  • 36.531568 tok/sPrompt (time to the first token) 64.3 s, 1 Lauf, kalt; 255,112 Tokens: 224 tok/s (1,137 s) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
TensorFold (bertholomus fork, branch ling-3.1-tp2 @ 72f16e48, bailing_v3 family on TensorFold v0.6.6 CUDA engine), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over direct RoCE, MTP drafting (adaptive depth)
Context
262,144 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required
  • tool-calling

Sources

Related recipes

2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

by fujitsupolycom

68tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking on, xhigh, with MTP3 (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B NVFP4 with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

67.4tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4 (nvidia, Experten) + FP8 n-gram/MTP, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-DraftsWeights Repo Updated Oct 26, 2026

Details
2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 EAQ 2.9 bpw with TensorFold on 2× DGX Spark

by MiaAI-Lab

63.9tok/sEveryday

1 request, prose prompt (temperature 0.7, thinking on), with DSpark (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 EAQ v1 2.9 bpw (Mia-AiLab; vision tower and DSpark draft blocks included)TensorFold native (Zig 0.17.0), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig @fbbc971) + recipe patches, TP=2 over ConnectX-7 (RoCE), DSpark draftsWeights Repo Updated Nov 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Con il vostro consenso utilizziamo Google Analytics (Google, USA) e la nostra misurazione dell’audience per capire come viene utilizzato il nostro sito. Le memorizzazioni strettamente necessarie, come la vostra scelta sui cookie, avvengono senza consenso. Potete modificare la vostra scelta in qualsiasi momento tramite «Impostazioni cookie» nel footer. Informativa sulla privacy · Informativa sui cookie