IT
Prenoti una chiamata

Local AI · Recipe · 4× DGX Spark

GLM-5.2 753.3B EXL3 3.5 bpw with vLLM on 4× DGX Spark

GLM-5.2 753.3B EXL3 3.5 bpw with vLLM on 4× DGX Spark: 20.2 tok/s according to github.com (dataset as of Sep 29, 2026).

by fujitsupolycom

20.2tok/sEveryday

1 request, realistic prompt, with MTP (fixed-4, greedy Draft) (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis34with thinkingno thinking 22
Instructions in the README Repo

Engine

Engine
vLLM
Quantization
EXL3 3.5 bpw gemischt (TR3v4), dynamisches NVFP4 MLA + FP8 RoPE KV (nvfp4_ds_mla), target-only exact Q40
Model family
GLM-5.2
Context
1,048,576
Parameters
753.3B MoE lt. zai-org/GLM-5.2 (HF-API Tensoren)
Creator
fujitsupolycom
GitHub stars
82
Repo updated
Sep 27, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)20.2 tok/scontext 16K decode · Einzelbeobachtung; unique cold prompts, temp 1.0 / top-p 0.95; Workload-Typ in der Quelle nicht klassifiziert Source 
  • Peak (decode)25.4 tok/sCoding Peak, Mittel über 5 normal requests (Median 25.57, Range 22.70–26.92), temp 1.0 Source 

Prefill by context

  • 16K671 tok/s16.386 Prompt-Tokens, TTFT 24.41 s Source 

What you need

Hardware
4 × NVIDIA DGX Spark (GB10)
Engine
vLLM
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview