IT
Prenoti una chiamata

Local AI · Recipe · 3× DGX Spark

GLM-5.3-Flash 320B EXL3 4 bpw with vLLM on 3× DGX Spark

GLM-5.3-Flash 320B EXL3 4 bpw with vLLM on 3× DGX Spark: 39.6 tok/s according to github.com (dataset as of Sep 29, 2026).

by MiaAI-Lab

39.6tok/sEveryday

1 request, prose prompt, with DFlash2 (per recipe)

Source: github.com
Intelligence (original model) · Artificial Analysis42with thinking

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
vLLM
Quantization
EXL3/TR3 4 bpw (routed experts), FP8 KV (fp8_ds_mla)
Model family
GLM-5.3-Flash
Context
1,000,000
Parameters
320B-A18B
Creator
MiaAI-Lab
GitHub stars
651
Repo updated
Sep 26, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)39.6 tok/sLab-Median 2026-09-14, temp 0, 400 tok, median of 3 Source 
  • Peak (decode)87.8 tok/s„structured“ = count-Prompt, Lab-Median, 400 tok Source 

What you need

Hardware
3 × NVIDIA DGX Spark (GB10)
Weights
Engine
vLLM
Context
1,000,000 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • Non-commercial
  • License unclear

Sources

Related recipes

2× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

by MiaAI-Lab

97.6tok/sEveryday

HumanEval 97.6% / GSM8K 98.0% with FP8 KV and 4-bit dense Source 

Intelligence42with thinking
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • Non-commercial
  • License unclear

EXL3/TR3 4bpw routed experts + 4-bit dense, BF16 elsewhere, FP8 KVTensorFold v0.5.0 (52 patches: DFlash2+copy drafts, 4-bit dense, FP8 KV, RoCE one-shot all-gather, shared KV pool, vision, tool calling)Weights Repo Updated Sep 30, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview