FR
Réserver un appel

Local AI · Recipe · 2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with vLLM on 2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with vLLM on 2× DGX Spark: 28.3 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

28.3tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
vLLM TP2 (image beam-exl3-gb10 via gb10/build-image.sh: Reflection arm64 Beam vLLM image + vllm-exl3 d3cfd394 with Diffbot Beam patch, ExLlamaV3 5be88657 for sm_121, kernels xmoe_dense/xmoe_do8/x8rh/f8rh), DFlash2 drafter (3 draft tokens)
Quantization
EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)
Model family
Beam-1.0-501B-A23B
Context
655,360
Parameters
501B MoE / 23B aktiv
Creator
bertholomus
GitHub stars
8
Repo updated
Oct 29, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)28.3 tok/scontext ~2K tok · VLLM-Spalte der Vergleichstabelle (gb10-Lane mit ausgelieferten Defaults, 655K-Fenster), 1 Stream, Prosa ~2K, T=0, Median aus 3 Läufen, DFlash2; eigene vLLM-Tabelle (400K-Fenster): 28.9 tok/s [28.1-30.2] Source 
  • Peak (decode)33.4 tok/scontext ~2K tok · VLLM-Spalte, Decode, 1 stream, code ~2K, Median aus 3 Läufen, DFlash2; 4 Streams Aggregat 61.0 tok/s Source 

Prefill by context

  • ~46K tok1,362 tok/sVLLM-Spalte, cold prefill, prose ~46K, frische Prompts (kein Prefix-Cache-Treffer), Median aus 3 Läufen Source 

Time to first token

Shorter is better.

  • Prompt not stated440 msVLLM-Spalte, short chat, first token, idle server (0.44 s) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM TP2 (image beam-exl3-gb10 via gb10/build-image.sh: Reflection arm64 Beam vLLM image + vllm-exl3 d3cfd394 with Diffbot Beam patch, ExLlamaV3 5be88657 for sm_121, kernels xmoe_dense/xmoe_do8/x8rh/f8rh), DFlash2 drafter (3 draft tokens)
Context
655,360 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • tool-calling

Sources

Related recipes

2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with TensorFold on 2× DGX Spark

by bertholomus

30.6tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required

EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)TensorFold (bertholomus fork, branch beam-1.0-tp2 @ d00201b4), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over 200 Gb/s RoCE, DFlash2 drafter (5 draft tokens, adaptive 3-5)Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

by fujitsupolycom

68tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking on, xhigh, with MTP3 (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next 125B NVFP4 with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

67.4tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVFP4 (nvidia, Experten) + FP8 n-gram/MTP, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-DraftsWeights Repo Updated Oct 26, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Avec votre consentement, nous utilisons Google Analytics (Google, États-Unis) et notre propre mesure d’audience pour comprendre l’utilisation de notre site. Les stockages strictement nécessaires, comme votre choix de cookies, se font sans consentement. Vous pouvez modifier votre choix à tout moment via « Paramètres des cookies » en pied de page. Politique de confidentialité · Politique relative aux cookies