EN
Book a call

Local AI · Recipe · 2× DGX Spark

GLM-5.3-Flash NVFP4 with SparkRing on 2× DGX Spark

GLM-5.3-Flash NVFP4 with SparkRing on 2× DGX Spark: 36 tok/s according to github.com (dataset as of Sep 29, 2026).

by fujitsupolycom

36tok/sEveryday

1 request, decode at 16K context (profile table), thinking always, max; peak C=8 (profile serves max 8 requests)

Source: github.com
Intelligence (original model) · Artificial Analysis42with thinking
Instructions in the README Repo

Engine

Engine
vLLM (local-inference-lab Fork) via SparkRing (switchless NCCL ueber ConnectX-7)
Quantization
NVFP4 Spark (Local Inference Lab)
Model family
GLM-5.3-Flash
Context
—
Parameters
165,5 GB Safetensors
Creator
fujitsupolycom
GitHub stars
91
Repo updated
Oct 6, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)36 tok/scontext 16K · 1 Nutzer, je 16K Kontext; Thinking immer an, max; Peak C=8 (Profil dient max. 8 Requests) Source 

Decode · parallel requests

Sums over all streams are marked; per request it is correspondingly less.

  • 16 parallel requests100 tok/sPeak (decode) · context 16K · Thinking always, max; peak C=8 (profile serves max 8 requests) Source 

Prefill by context

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Engine
vLLM (local-inference-lab Fork) via SparkRing (switchless NCCL ueber ConnectX-7)

Notes

What matters before you rebuild it.

  • Custom kernel required

Sources

Related recipes

2× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

by MiaAI-Lab

60.4tok/sEveryday

Prose decode, 1 request; quality (FP8 KV, 4-bit dense): GSM8K 98.8% / HumanEval 95.7% / HumanEval+ & MBPP+ 86.5% Source 

Intelligence42with thinking
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • Non-commercial
  • License unclear

EXL3/TR3 4bpw routed experts + 4-bit dense, BF16 elsewhere, FP8 KVTensorFold v0.6.0 + Recipe v1.10 (README: 96 Patches; gepinntes Image v0.6.0-e66c657a433d mit 107 Patches: DFlash2+Copy-Drafts, 4-bit dense, FP8-KV, RoCE-One-Shot-All-Gather, Shared KV-Pool, optionaler Spill-Tier, Vision, Tool-Calling, Anthropic /v1/messages; PARALLEL=8 opt-in)Weights Repo Updated Oct 10, 2026

Details
2× DGX Spark

GLM-5.3-Flash 320B-A18B EXL3 4 bpw with ExLlamaV3 on 2× DGX Spark

by MiaAI-Lab

36.1tok/sEveryday

1 request, prose prompt, with DFlash2 (per recipe) Source 

Intelligence42with thinking
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • Non-commercial
  • License unclear

EXL3 4bpw (TR3) + FP8 KV (fp8_ds_mla)TensorFold-basiertes Kit (ghcr.io/miaai-lab/glm-5.3-flash-2x-dgx-sparks:exl3-instanttensor), DFlash2 7 draft tokens; opt-in Speed-Boost-Profil (einmaliger ~25-min-Build auf dem Head)Weights Repo Updated Oct 4, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview