Local AI · Recipe · 4× DGX Spark

GLM-5.3-Flash EXL3 4bpw with TensorFold on 4× DGX Spark

GLM-5.3-Flash EXL3 4bpw with TensorFold on 4× DGX Spark: 65.4 tok/s according to github.com (dataset as of Nov 3, 2026).

by tonyd2wild

65.4tok/sEveryday

1 request, prose prompt, with MTP + DFlash2 (per recipe)

Source: github.com

Intelligence (original model) · Artificial Analysis42with thinking

Instructions in the README Repo

Engine: TensorFold v0.6.0 + MiaAI-Lab GLM-5.3-Flash patch series (engine-multi, + tonyd2wild changes)
Quantization: EXL3 4bpw routed experts (mcg) + q4 dense (at load) + FP8 KV
Model family: GLM-5.3-Flash
Context: 500,000
Creator: tonyd2wild
GitHub stars: 3
Repo updated: Oct 12, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

Everyday (decode)65.4 tok/sProse, Single Stream, Median von 3, temp 0, reasoning_effort low, MTP + DFlash2 + Copy-Drafts, TP4-Lane (Default) Source
Mixed (decode)124 tok/sGemischte reale Prompts (rotiert, ohne Zählen), C1-Aggregat = 1 Stream, Median von 3 Runden Source
Peak (decode)180 tok/sCount to 100 (repetitiv), Single Stream, Median von 3 Source

Prefill by context

113.9112,808 tok/sCold (salted prompt, 1 Output-Token), TTFT 40.6 s Source

Time to first token

Shorter is better.

Prompt not stated170 msProse, Single Stream Source

What you need

Hardware: 4 × NVIDIA DGX Spark (GB10)
Weights: Main weightsbrandonmusic/GLM-5.3-Flash-tr3-4bpw other
Basezai-org/GLM-5.3-Flash-BF16 MIT
Draft modelincoai/GLM-5.3-Flash-DFlash2 Non-commercial · CC-BY-NC-ND-4.0
Engine: TensorFold v0.6.0 + MiaAI-Lab GLM-5.3-Flash patch series (engine-multi, + tonyd2wild changes)
Context: 500,000 tokens

Notes

What matters before you rebuild it.

Custom kernel required
vision
tool-calling
Non-commercial
License unclear

Sources

Related recipes

4× DGX Spark

GLM-5.3-Flash NVFP4/MXFP8 CSF with SparkRing on 4× DGX Spark

by fujitsupolycom

68tok/sEveryday

1 request, generation at 32K context (speed table; measured only as a path of four Sparks inside a ring of eight, not as a ring of four), thinking always on (measured before the profile default high, i.e. template default max), with MTP3 (per recipe) Source

Intelligence42with thinking

Custom kernel required
License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

2× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

by MiaAI-Lab

60.4tok/sEveryday

Prose decode, 1 request; quality (FP8 KV, 4-bit dense): GSM8K 98.8% / HumanEval 95.7% / HumanEval+ & MBPP+ 86.5% Source

Intelligence42with thinking

≤ 3 bit: quantization may cost quality
Custom kernel required
Non-commercial
License unclear

EXL3/TR3 4bpw routed experts + 4-bit dense, BF16 elsewhere, FP8 KVTensorFold v0.6.0 + Recipe v1.10 (README: 96 Patches; gepinntes Image v0.6.0-e66c657a433d mit 107 Patches: DFlash2+Copy-Drafts, 4-bit dense, FP8-KV, RoCE-One-Shot-All-Gather, Shared KV-Pool, optionaler Spill-Tier, Vision, Tool-Calling, Anthropic /v1/messages; PARALLEL=8 opt-in)Weights Repo Updated Oct 25, 2026

Details

2× DGX Spark

GLM-5.3-Flash EXL3 4bpw with TensorFold on 2× DGX Spark

by tonyd2wild

44tok/sEveryday

1 request, prose prompt, with MTP + DFlash2 (per recipe) Source

Intelligence42with thinking

Custom kernel required
Non-commercial
License unclear

EXL3 4bpw routed experts (mcg) + q4 dense (at load) + FP8 KVTensorFold v0.6.0 + MiaAI-Lab GLM-5.3-Flash patch series (engine-multi, + tonyd2wild changes)Weights Repo Updated Oct 12, 2026

Details

4× DGX Spark

GLM-5.3-Flash 320B-A18B NVFP4 with vLLM on 4× DGX Spark

by tonyd2wild

40.8tok/sEveryday

1 request, prose prompt, with DFlash2 (per recipe) Source

Intelligence42with thinking

Custom kernel required
Non-commercial

NVFP4 weights + FP8 KV (Lane B: NVFP4 KV)vLLM 0.28.0Weights Repo Updated Oct 10, 2026

Details

3× DGX Spark

GLM-5.3-Flash 320B EXL3 4 bpw with vLLM on 3× DGX Spark

by MiaAI-Lab

40.1tok/sEveryday

1 request, prose prompt, with DFlash2, TR3 weights (per recipe) Source

Intelligence42with thinking

≤ 3 bit: quantization may cost quality
Custom kernel required
Non-commercial
License unclear

EXL3/TR3 4 bpw (routed experts), FP8 KV (fp8_ds_mla)vLLMWeights Repo Updated Oct 18, 2026

Details

2× DGX Spark

GLM-5.3-Flash NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

by fujitsupolycom

38tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking always on (measured before the profile default high, i.e. template default max), with MTP3 (per recipe) Source

Intelligence42with thinking

Custom kernel required
License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshop Back to overview

Engine

Measurements

Decode · 1 request

Prefill by context

Time to first token

What you need

Notes

Sources

Related recipes

GLM-5.3-Flash NVFP4/MXFP8 CSF with SparkRing on 4× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

GLM-5.3-Flash EXL3 4bpw with TensorFold on 2× DGX Spark

GLM-5.3-Flash 320B-A18B NVFP4 with vLLM on 4× DGX Spark

GLM-5.3-Flash 320B EXL3 4 bpw with vLLM on 3× DGX Spark

GLM-5.3-Flash NVFP4/MXFP8 CSF with SparkRing on 2× DGX Spark

Local AI in your company?