FR
Réserver un appel

Local AI · Recipe · 4× DGX Spark

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark: ~36 tok/s according to github.com (dataset as of Nov 3, 2026).

by bertholomus

~36tok/sPeakapproximate · method not stated

1 request, prompt type not stated, range 33–39 tok/s, with MTP-3 (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis45with thinking

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
TensorFold 0.6.0 + bertholomus glm-dsa-tp4 branch (release numbers on 3eb35dd, head ddfad35), image nvcr.io/nvidia/pytorch:26.07-py3, TP4 over direct RoCE, MTP-3 drafting, config B (1M window, DCP4, --parallel 4)
Quantization
EXL3 3.0 bpw (exllamav3 1.5.3; routed experts 3.0, attention/shared/dense ~5, MTP experts 4, lm_head 6; 3.04 bpw avg), Q5 latent cache
Model family
GLM-5.3
Context
1,048,576
Parameters
753B MoE
Creator
bertholomus
GitHub stars
4
Repo updated
Oct 9, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Peak (decode)36 tok/sConfig B (1M-Fenster, DCP4, --parallel 4), single stream 33-39 tok/s (Mittelwert des Bereichs), greedy, MTP-3, HTTP inkl. Prompt-Zeit; ohne --parallel 36.7-43.3; Prompt-Typ nicht genannt; 4 concurrent code streams ~64 tok/s aggregate Source 

Prefill by context

  • 32K402 tok/sConfig B, prefill 402 tok/s at 32K; first token ~76 s at ~31K Source 

What you need

Hardware
4 × NVIDIA DGX Spark (GB10)
Engine
TensorFold 0.6.0 + bertholomus glm-dsa-tp4 branch (release numbers on 3eb35dd, head ddfad35), image nvcr.io/nvidia/pytorch:26.07-py3, TP4 over direct RoCE, MTP-3 drafting, config B (1M window, DCP4, --parallel 4)
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • tool-calling
  • License unclear

Sources

Related recipes

8× DGX Spark

GLM-5.3 NVFP4 with SparkRing on 8× DGX Spark

by fujitsupolycom

46tok/sEveryday

1 request, generation at 32K context (speed table, ring of eight, TP8/DCP4), thinking always on (measured before the profile default high, i.e. template default max), with MTP2 (per recipe); experimental profile Source 

Intelligence45with thinking
  • Custom kernel required
  • Experimental

NVFP4-Experten/BF16 (Local Inference Lab; Dense-Linears beim Laden MXFP8)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
3× DGX Spark

GLM-5.3 753B EXL3 2.75 bpw with TensorFold on 3× DGX Spark

by MiaAI-Lab

31.3tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source 

Intelligence45with thinking
  • Custom kernel required
  • License unclear

EXL3 2,75 bpw Ø (Experten 2–4 bit), 4-bit dense, BF16 sonst, FP4 KV (Default CP=1)TensorFold v0.6.0 + 110 Patches, Image ghcr.io/miaai-lab/glm-5.3-exl3-3x-dgx-sparks-tensorfold:v0.6.0-4f3223245926, DSpark + Copy-Drafts, Context Parallelism, RoCEWeights Repo Updated Oct 4, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Avec votre consentement, nous utilisons Google Analytics (Google, États-Unis) et notre propre mesure d’audience pour comprendre l’utilisation de notre site. Les stockages strictement nécessaires, comme votre choix de cookies, se font sans consentement. Vous pouvez modifier votre choix à tout moment via « Paramètres des cookies » en pied de page. Politique de confidentialité · Politique relative aux cookies