Local AI · Recipe · 1× Mac

GLM-5.3 UD-IQ2_XXS with DwarfStar on 1× Mac

GLM-5.3 UD-IQ2_XXS with DwarfStar on 1× Mac: 6.14 tok/s according to github.com (dataset as of Nov 3, 2026).

6.14tok/sMixed

1 request, mixed prompt set, context 1k, no speculative decoding

Intelligence (original model) · Artificial Analysis45with thinkingOriginal model's value. Heavily compressed (~2.1 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine: DwarfStar (ds4) main@0aaea5a (2026-09-20)
Quantization: UD-IQ2_XXS routed, blk78 Q2K, 196.58 GiB
Model family: GLM-5.3
Context: —
Creator: antirez
GitHub stars: 24,199
Repo updated: Nov 2, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

Mixed (decode)6.14 tok/scontext 1K · Median, 16 GiB Budget, 64 Tokens Source
Peak (decode)6.14 tok/scontext 1K · Median, 16 GiB Budget, 64 Tokens Source

Prefill by context

8K91 tok/sAbout; 16K ctx allocation, 35.36 GiB Expert-Cache, Snapshot-Restore, 16 Output-Tokens Source

What you need

Hardware: 1 × Mac / Apple Silicon
Weights: Main weightsantirez/glm-5.3-gguf License unclear
Engine: DwarfStar (ds4) main@0aaea5a (2026-09-20)

Notes

What matters before you rebuild it.

≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
Streams from SSD
License unclear

Sources

Related recipes

8× DGX Spark

GLM-5.3 NVFP4 with SparkRing on 8× DGX Spark

by fujitsupolycom

46tok/sEveryday

1 request, generation at 32K context (speed table, ring of eight, TP8/DCP4), thinking always on (measured before the profile default high, i.e. template default max), with MTP2 (per recipe); experimental profile Source

Intelligence45with thinking

Custom kernel required
Experimental

NVFP4-Experten/BF16 (Local Inference Lab; Dense-Linears beim Laden MXFP8)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

4× DGX Spark

GLM-5.3 EXL3 3.38 bpw with TensorFold on 4× DGX Spark

by vcruz305

35.7tok/sEveryday

1 request, prose prompt, with DFlash2, thinking on (per recipe) Source

Intelligence45with thinking

Custom kernel required
Non-commercial

EXL3 SAGE MixedK 3.38 bpw (319 GB; serviert mit BF16-lm_head), bf16 KV, Draft DFlash2 d7/c0.60TensorFold Fork vcruz305/TensorFold glm53-tp4-spark @757a851 (PR #159, TP=4) + b12x RoCE (b58f34e)Weights Repo Updated Oct 5, 2026

Details

3× DGX Spark

GLM-5.3 753B EXL3 2.75 bpw with TensorFold on 3× DGX Spark

by MiaAI-Lab

31.3tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source

Intelligence45with thinking

Custom kernel required
License unclear

EXL3 2,75 bpw Ø (Experten 2–4 bit), 4-bit dense, BF16 sonst, FP4 KV (Default CP=1)TensorFold v0.6.0 + 110 Patches, Image ghcr.io/miaai-lab/glm-5.3-exl3-3x-dgx-sparks-tensorfold:v0.6.0-4f3223245926, DSpark + Copy-Drafts, Context Parallelism, RoCEWeights Repo Updated Oct 4, 2026

Details

4× DGX Spark

GLM-5.3 743B NVFP4 with vLLM on 4× DGX Spark

by tonyd2wild

53.3tok/sPeak

1 request, structured output end-to-end (count-to-300); DCP4 lane 2: NVFP4 KV Source

Intelligence45with thinking

Custom kernel required
License unclear

Int4-Int8Mix + NVFP4 KVvLLM 0.23.1rc1.dev190 + DCP4 (Decode-Context-Parallelism, 2 Lanes, ajclark-Patches, vendored dcp/) + DFlash2 (k=7)Weights Repo Updated Oct 6, 2026

Details

4× DGX Spark

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

~36tok/sPeakapproximate · method not stated

1 request, prompt type not stated, range 33–39 tok/s, with MTP-3 (per recipe) Source

Intelligence45with thinking

≤ 3 bit: quantization may cost quality
Custom kernel required
License unclear

EXL3 3.0 bpw (exllamav3 1.5.3; routed experts 3.0, attention/shared/dense ~5, MTP experts 4, lm_head 6; 3.04 bpw avg), Q5 latent cacheTensorFold 0.6.0 + bertholomus glm-dsa-tp4 branch (release numbers on 3eb35dd, head ddfad35), image nvcr.io/nvidia/pytorch:26.07-py3, TP4 over direct RoCE, MTP-3 drafting, config B (1M window, DCP4, --parallel 4)Weights Repo Updated Oct 9, 2026

Details

1× Mac