Local AI · Recipe · 4× DGX Spark

GLM-5.3 743B NVFP4 with vLLM on 4× DGX Spark

GLM-5.3 743B NVFP4 with vLLM on 4× DGX Spark: 53.3 tok/s according to github.com (dataset as of Nov 3, 2026).

by tonyd2wild

53.3tok/sPeak

1 request, structured output end-to-end (count-to-300); DCP4 lane 2: NVFP4 KV

Source: github.com

Intelligence (original model) · Artificial Analysis45with thinking

Instructions in the README Repo

Engine: vLLM 0.23.1rc1.dev190 + DCP4 (Decode-Context-Parallelism, 2 Lanes, ajclark-Patches, vendored dcp/) + DFlash2 (k=7)
Quantization: Int4-Int8Mix + NVFP4 KV
Model family: GLM-5.3
Context: 524,288
Parameters: 743B total / ~40B active
Creator: tonyd2wild
GitHub stars: 17
Repo updated: Oct 6, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

Peak (decode)53.3 tok/sStructured output, end-to-end (Count-to-300); DCP4: Lane 2 (NVFP4-KV, 689.772-Token-KV-Pool @262K Kontext) und Lane 1 (fp8-KV, 462.308) decodieren mit gleichem Tempo; Tool-Call-Prompt 57,6; Code 39,9; JSON 36,0; Needle 3/3 @490.425 Token (524.288-Fenster); Prefill-Trade-off ~400 tok/s Source

What you need

Hardware: 4 × NVIDIA DGX Spark (GB10)
Weights: Main weightsTech2wild/GLM-5.3-Int4-Int8Mix other
Basezai-org/GLM-5.3-BF16 other
Engine: vLLM 0.23.1rc1.dev190 + DCP4 (Decode-Context-Parallelism, 2 Lanes, ajclark-Patches, vendored dcp/) + DFlash2 (k=7)
Context: 524,288 tokens

Notes

What matters before you rebuild it.

Custom kernel required
License unclear

Sources

Related recipes

8× DGX Spark

GLM-5.3 NVFP4 with SparkRing on 8× DGX Spark

by fujitsupolycom

46tok/sEveryday

1 request, generation at 32K context (speed table, ring of eight, TP8/DCP4), thinking always on (measured before the profile default high, i.e. template default max), with MTP2 (per recipe); experimental profile Source

Intelligence45with thinking

Custom kernel required
Experimental

NVFP4-Experten/BF16 (Local Inference Lab; Dense-Linears beim Laden MXFP8)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

4× DGX Spark

GLM-5.3 EXL3 3.38 bpw with TensorFold on 4× DGX Spark

by vcruz305

35.7tok/sEveryday

1 request, prose prompt, with DFlash2, thinking on (per recipe) Source

Intelligence45with thinking

Custom kernel required
Non-commercial

EXL3 SAGE MixedK 3.38 bpw (319 GB; serviert mit BF16-lm_head), bf16 KV, Draft DFlash2 d7/c0.60TensorFold Fork vcruz305/TensorFold glm53-tp4-spark @757a851 (PR #159, TP=4) + b12x RoCE (b58f34e)Weights Repo Updated Oct 5, 2026

Details

3× DGX Spark

GLM-5.3 753B EXL3 2.75 bpw with TensorFold on 3× DGX Spark

by MiaAI-Lab

31.3tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source

Intelligence45with thinking

Custom kernel required
License unclear

EXL3 2,75 bpw Ø (Experten 2–4 bit), 4-bit dense, BF16 sonst, FP4 KV (Default CP=1)TensorFold v0.6.0 + 110 Patches, Image ghcr.io/miaai-lab/glm-5.3-exl3-3x-dgx-sparks-tensorfold:v0.6.0-4f3223245926, DSpark + Copy-Drafts, Context Parallelism, RoCEWeights Repo Updated Oct 4, 2026

Details

1× Mac

GLM-5.3 UD-IQ2_XXS with DwarfStar on 1× Mac

by antirez

6.14tok/sMixed

1 request, mixed prompt set, context 1k, no speculative decoding Source

Intelligence45with thinkingOriginal model's value. Heavily compressed (~2.1 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality
Streams from SSD
License unclear

UD-IQ2_XXS routed, blk78 Q2K, 196.58 GiBDwarfStar (ds4) main@0aaea5a (2026-09-20)Weights Repo Updated Nov 2, 2026

Details

4× DGX Spark

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

~36tok/sPeakapproximate · method not stated

1 request, prompt type not stated, range 33–39 tok/s, with MTP-3 (per recipe) Source

Intelligence45with thinking

≤ 3 bit: quantization may cost quality
Custom kernel required
License unclear

EXL3 3.0 bpw (exllamav3 1.5.3; routed experts 3.0, attention/shared/dense ~5, MTP experts 4, lm_head 6; 3.04 bpw avg), Q5 latent cacheTensorFold 0.6.0 + bertholomus glm-dsa-tp4 branch (release numbers on 3eb35dd, head ddfad35), image nvcr.io/nvidia/pytorch:26.07-py3, TP4 over direct RoCE, MTP-3 drafting, config B (1M window, DCP4, --parallel 4)Weights Repo Updated Oct 9, 2026

Details

4× DGX Spark

DeepSeek-V4.1-Flash MXFP4 with SGLang on 4× DGX Spark (MiaAI-Lab)

by MiaAI-Lab

87.7tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source

Intelligence40with thinking no thinking 25

MXFP4 experts + FP8 dense, Engram-Tabellen auf NVMeSGLangWeights Repo Updated Oct 17, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshop Back to overview

Engine

Measurements

Decode · 1 request

What you need

Notes

Sources

Related recipes

GLM-5.3 NVFP4 with SparkRing on 8× DGX Spark

GLM-5.3 EXL3 3.38 bpw with TensorFold on 4× DGX Spark

GLM-5.3 753B EXL3 2.75 bpw with TensorFold on 3× DGX Spark

GLM-5.3 UD-IQ2_XXS with DwarfStar on 1× Mac

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark

DeepSeek-V4.1-Flash MXFP4 with SGLang on 4× DGX Spark (MiaAI-Lab)

Local AI in your company?