---
type: "Article"
title: "GLM-5.3-Flash FP4-Local (NVFP4/MXFP8) with vLLM on 2× DGX Spark"
description: "GLM-5.3-Flash FP4-Local (NVFP4/MXFP8) with vLLM on 2× DGX Spark: recipe by myllmbox with speed, condition and source for every number, weights and license."
resource: "https://www.contextstudios.ai/local-ai/myllmbox--glm53-flash-fp4-local-2x-dgx-spark"
language: "en"
generated:
  by: "process:contextstudios-md/1"
  at: "2026-11-11T03:20:43.494Z"
status: "stable"
---

# GLM-5.3-Flash FP4-Local (NVFP4/MXFP8) with vLLM on 2× DGX Spark

GLM-5.3-Flash FP4-Local (NVFP4/MXFP8) with vLLM on 2× DGX Spark: 34.4 tok/s according to github.com (dataset as of Nov 3, 2026).

- Creator: [myllmbox](https://www.contextstudios.ai/local-ai/creators/myllmbox.md)
- Engine: vLLM v0.32.0 (Image myllmbox/glm53-flash-cluster-vllm:v3: mbx-Loader, NVFP4-Experten auf Marlin, dynamische MTP-Draft-Tiefe 1–9, One-Shot-RoCE-All-Reduce via b12x), TP=2 über RDMA
- Quantization: NVIDIA FP4-Local-Checkpoint wie ausgeliefert (NVFP4-Experten, MXFP8-Dense-Layer); KV-Pool 1.145.238 Token (12G/Box); Marlin MoE
- Model family: GLM-5.3-Flash
- Hardware: dgx-spark x2
- Context: 400000
- Artificial Analysis Intelligence Index: 41.8 ([source](https://artificialanalysis.ai/models/glm-5-3-flash))

## All numbers

- 34.4 tok/s (Everyday; Everyday, 1 request, prose prompt (7,000-word story), with MTP (per recipe)) — [source](https://github.com/myllmbox/glm53-flash-cluster-recipe/blob/main/README.md)
- decode_tps: 34.4 (1 request) — [source](https://github.com/myllmbox/glm53-flash-cluster-recipe/blob/main/README.md)
- decode_tps: 86.7 (1 request) — [source](https://github.com/myllmbox/glm53-flash-cluster-recipe/blob/main/README.md)
- prefill_tps: 1836 (1 request) — [source](https://github.com/myllmbox/glm53-flash-cluster-recipe/blob/main/README.md)
- ttft_ms: 1130 (1 request) — [source](https://github.com/myllmbox/glm53-flash-cluster-recipe/blob/main/README.md)

## Weights

- [nvidia/GLM-5.3-Flash-FP4-Local](https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local) (main, other)
- [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) (base, mit)

[https://github.com/myllmbox/glm53-flash-cluster-recipe](https://github.com/myllmbox/glm53-flash-cluster-recipe)

## Related

- [Local AI on your hardware.](https://www.contextstudios.ai/local-ai.md)