---
type: "Article"
title: "GLM-5.3-Flash TrellisMX codec-v2 FP8 with vLLM on 4× RTX PRO 6000"
description: "GLM-5.3-Flash TrellisMX codec-v2 FP8 with vLLM on 4× RTX PRO 6000: recipe by brandonmusic with speed, condition and source for every number, weights and license."
resource: "https://www.contextstudios.ai/local-ai/brandonmusic--glm53-flash-tr3-fp8-codecv2-4x-rtx6000"
language: "en"
generated:
  by: "process:contextstudios-md/1"
  at: "2026-11-11T03:20:20.903Z"
status: "stable"
---

# GLM-5.3-Flash TrellisMX codec-v2 FP8 with vLLM on 4× RTX PRO 6000

GLM-5.3-Flash TrellisMX codec-v2 FP8 with vLLM on 4× RTX PRO 6000: 189 tok/s according to huggingface.co (dataset as of Nov 3, 2026).

- Creator: [brandonmusic](https://www.contextstudios.ai/local-ai/creators/brandonmusic.md)
- Engine: vLLM (TrellisMX-Runtime, Image verdictai/trellismx:glm53-codecv2-prefill-20260927 @sha256:6240b05f…; TP4/DCP4, MTP3)
- Quantization: TrellisMX codec-v2 K4 (4,25 bpw Payload, Routed- + MTP-Experten) → native FP8 E4M3 Experten-Math (UE8M0/K32), FP8 KV
- Model family: GLM-5.3-Flash
- Hardware: rtx-pro-6000 x4
- Context: 1000000
- Artificial Analysis Intelligence Index: 41.8 ([source](https://artificialanalysis.ai/models/glm-5-3-flash))

## All numbers

- 189 tok/s (Peak; Peak, 1 request, context 0 (78 prompt tokens), synthetic sustained decode with ignore-EOS, with MTP3 (per recipe)) — [source](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8#reproduce-the-runtime)
- decode_tps: 189.2 (1 request) — [source](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8#reproduce-the-runtime)
- prefill_tps: 9520 (1 request) — [source](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8#reproduce-the-runtime)
- ttft_ms: 13751 (1 request) — [source](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8#reproduce-the-runtime)

## Weights

- [brandonmusic/glm-5.3-flash-tr3-fp8](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8) (main, mit)
- [zai-org/GLM-5.3-Flash-BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16) (base, mit)

[https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8](https://huggingface.co/brandonmusic/glm-5.3-flash-tr3-fp8)

## Related

- [Local AI on your hardware.](https://www.contextstudios.ai/local-ai.md)