EN
Book a call

Local AI · Recipe · 2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with TensorFold on 2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with TensorFold on 2× DGX Spark: 30.6 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

30.6tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
TensorFold (bertholomus fork, branch beam-1.0-tp2 @ d00201b4), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over 200 Gb/s RoCE, DFlash2 drafter (5 draft tokens, adaptive 3-5)
Quantization
EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)
Model family
Beam-1.0-501B-A23B
Context
868,352
Parameters
501B MoE / 23B aktiv
Creator
bertholomus
GitHub stars
8
Repo updated
Oct 29, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)30.6 tok/scontext ~2K tok · 1 Stream, Prosa ~2K (Wortsuppe-Dokument + technische Schreibaufgabe, gb10/rows.py), T=0, thinking off, Median aus 3 Läufen, DFlash2 Source 
  • Peak (decode)45.5 tok/scontext ~2K tok · Decode, 1 stream, code ~2K (realer Code + 'explain what this code does'), Median aus 3 Läufen, DFlash2; 4 Streams Aggregat 64.7 tok/s Source 

Prefill by context

  • ~46K tok1,390 tok/sCold prefill, prose ~46K, frische Prompts, prompt tokens / time to first token, Median aus 3 Läufen Source 

Time to first token

Shorter is better.

  • Prompt not stated390 msShort chat, first token, idle server (0.39 s), Streaming Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
TensorFold (bertholomus fork, branch beam-1.0-tp2 @ d00201b4), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over 200 Gb/s RoCE, DFlash2 drafter (5 draft tokens, adaptive 3-5)
Context
868,352 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • tool-calling

Sources

Related recipes

2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with vLLM on 2× DGX Spark

by bertholomus

28.3tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)vLLM TP2 (image beam-exl3-gb10 via gb10/build-image.sh: Reflection arm64 Beam vLLM image + vllm-exl3 d3cfd394 with Diffbot Beam patch, ExLlamaV3 5be88657 for sm_121, kernels xmoe_dense/xmoe_do8/x8rh/f8rh), DFlash2 drafter (3 draft tokens)Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

by fujitsupolycom

68tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking on, xhigh, with MTP3 (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B NVFP4 with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

67.4tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4 (nvidia, Experten) + FP8 n-gram/MTP, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-DraftsWeights Repo Updated Oct 26, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy