Skip to content

EN

Local AI · Recipe · 2× Mac

GLM-5.2 UD-IQ2_XXS with DwarfStar on 2× Mac

GLM-5.2 UD-IQ2_XXS with DwarfStar on 2× Mac: 16.7 tok/s according to github.com (dataset as of Nov 3, 2026).

16.7tok/sPeak

1 request, best value reported by source, context 4k

Source: github.com

Intelligence (original model) · Artificial Analysis34with thinking no thinking 22Original model's value. Heavily compressed (~2.1 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine: DwarfStar (ds4) main@0aaea5a (2026-09-20)
Quantization: UD-IQ2_XXS routed, 196.58 GiB
Model family: GLM-5.2
Context: —
Creator: antirez
GitHub stars: 24,199
Repo updated: Nov 2, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

Peak (decode)16.7 tok/scontext 4K · About, 256 teacher-forced tokens Source

Prefill by context

4K214 tok/sAbout Source

What you need

Hardware: 2 × Mac / Apple Silicon
Weights: Main weightsantirez/glm-5.2-gguf License unclear
Engine: DwarfStar (ds4) main@0aaea5a (2026-09-20)

Notes

What matters before you rebuild it.

≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
Multiple nodes
RDMA link
License unclear

Sources

Related recipes

4× RTX PRO 6000

GLM-5.2 EXL3 3.0 bpw with vLLM on 4× RTX PRO 6000 (400k context, text)

81.7tok/sEveryday

1 request, prose prompt, with MTP Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality

EXL3 TR3 3.0 bpw + NVFP4 DSA-MLA KVvLLM (EXL3/Trellis-Runtime) local/glm52-exl3-vision:v22-dcpstrideWeights Repo Updated Aug 1, 2026

4× RTX PRO 6000

GLM-5.2 EXL3 3.0 bpw with vLLM on 4× RTX PRO 6000 (400k context, vision)

70.7tok/sEveryday

1 request, prose prompt, with MTP Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality

EXL3 TR3 3.0 bpw + BF16 MoonViT/PatchMerger, FP8 MLA KVvLLM (EXL3/Trellis-Runtime) local/glm52-exl3-vision:v21Weights Repo Updated Sep 1, 2026

4× RTX PRO 6000

GLM-5.2 NVFP4 with vLLM on 4× RTX PRO 6000 (192k context)

by brandonmmusic-max

64tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality
Custom kernel required

MXFP8/NVFP4/NF3-Hybrid weights + NVFP4 MLA KVvLLMWeights Repo Updated Sep 6, 2026

4× RTX PRO 6000

GLM-5.2 NVFP4 with vLLM on 4× RTX PRO 6000 (250k context)

by brandonmmusic-max

58.5tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source

Intelligenceno independent value

Custom kernel required

NVFP4 (REAP 172 Experten/Layer) + NVFP4 MLA KV (nvfp4_ds_mla)vLLMWeights Repo Updated Jun 29, 2026

4× RTX PRO 6000

GLM-5.2 EXL3 3.0 bpw with vLLM on 4× RTX PRO 6000 (500k context, text)

104tok/sMixed

1 request, mixed prompt set, with MTP (per recipe) Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality

EXL3/Trellis 3.0 bpw, nvfp4_ds_mla KVvLLMWeights Repo Updated Jul 28, 2026

4× RTX PRO 6000

GLM-5.2 504B NVFP4 with vLLM on 4× RTX PRO 6000

92.5tok/sMixed

1 request, mixed prompt set, with MTP Source

Intelligenceno independent value

NVFP4 (modelopt) Experten, BF16 Router/Attention, FP8 KVvLLMWeights Repo Updated Jun 21, 2026

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshop Back to overview