Skip to content

EN

Local AI · Creator

0xSero

30 published recipes.

Recipes

4× RTX PRO 6000

GLM-5.2 EXL3 3.0 bpw with vLLM on 4× RTX PRO 6000 (400k context, text)

81.7tok/sEveryday

1 request, prose prompt, with MTP Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality

EXL3 TR3 3.0 bpw + NVFP4 DSA-MLA KVvLLM (EXL3/Trellis-Runtime) local/glm52-exl3-vision:v22-dcpstrideWeights Repo Updated Aug 1, 2026

4× RTX PRO 6000

GLM-5.2 EXL3 3.0 bpw with vLLM on 4× RTX PRO 6000 (400k context, vision)

70.7tok/sEveryday

1 request, prose prompt, with MTP Source

Intelligence34with thinking no thinking 22

≤ 3 bit: quantization may cost quality

EXL3 TR3 3.0 bpw + BF16 MoonViT/PatchMerger, FP8 MLA KVvLLM (EXL3/Trellis-Runtime) local/glm52-exl3-vision:v21Weights Repo Updated Sep 1, 2026

4× DGX Spark

DeepSeek-V4.1-Flash MXFP4 with SGLang on 4× DGX Spark (0xSero)

59.1tok/sEveryday

1 request, prose prompt, with DSpark (per recipe) Source

Intelligence40with thinking no thinking 25

MXFP4 experts + FP8 denseSGLangWeights Repo Updated Sep 26, 2026

1× Strix Halo

Qwen3.5-122B-A10B Q4_K_M with llama.cpp on 1× Strix Halo

28.4tok/sEveryday

1 request, prose prompt, no speculative decoding Source

Intelligenceno independent value

License unclear

GGUF Q4_K_M (4.86 bpw)llama.cpp b8746Weights Repo Updated May 30, 2026

8× RTX 3090Qwen3.5-397B-A17B (REAP 264B)

Qwen3.5-397B-A17B FP8 with vLLM on 8× RTX 3090

62tok/sMixed

1 request, mixed prompt set, no speculative decoding Source

Intelligenceno independent value

W4A16 + FP8 KVvLLM 0.19.0Weights Repo Updated May 30, 2026

4× RTX PRO 6000

GLM-5.1 478.4B NVFP4 with SGLang on 4× RTX PRO 6000

42.7tok/sMixed

1 request, mixed prompt set, no speculative decoding Source

Intelligenceno independent value

NVFP4 (modelopt_fp4, group 16)SGLang 0.5.10.post1Weights Repo Updated May 30, 2026

4× RTX 3090Qwen3.5-35B-A3B (REAP 28B)

Qwen3.5-35B-A3B BF16 with vLLM on 4× RTX 3090

12.5tok/sMixed

1 request, mixed prompt set, no speculative decoding Source

Intelligenceno independent value

BF16 (keine Quantisierung)vLLMWeights Repo Updated May 30, 2026

4× RTX PRO 6000

GLM-5.3-Flash 320B-A18B NVFP4 with SGLang on 4× RTX PRO 6000

208tok/sPeak

1 request, best value reported by source Source

Intelligence42with thinking

NVFP4 + fp8_e4m3 KVSGLangWeights Repo Updated Sep 17, 2026

4× RTX PRO 6000

DeepSeek-V4.1-Flash with vLLM on 4× RTX PRO 6000

202tok/sPeak

1 request, best value reported by source, context 200k Source

Intelligence40with thinking no thinking 25

Experimental

native Gewichte, modell-native komprimierte KVvLLMWeights Repo Updated Sep 24, 2026

4× RTX 3090

Qwen3.8 27B AWQ with SGLang on 4× RTX 3090

137tok/sPeak

1 request, best value reported by source, context 8k Source

Intelligence34with thinking no thinking 20

License unclear

AWQ-INT4 W4A16 (Marlin), bf16 KVSGLang commit 2e7c85da6Weights Repo Updated Sep 23, 2026

4× DGX Spark

GLM-5.3-Flash NVFP4 with vLLM on 4× DGX Spark

118tok/sPeak

1 request, JSON prompt Source

Intelligence42with thinking

Non-commercial

NVFP4 + FP8 KV (fp8_e4m3)vLLM 0.1.dev20051+g487ecf187Weights Repo Updated Sep 26, 2026

4× RTX PRO 6000

MiniMax-M3 428B-A23B MXFP4 with vLLM on 4× RTX PRO 6000

113tok/sPeak

1 request, best value reported by source Source

Intelligence29with thinking

License unclear

MXFP4, BF16 KVvLLM gepatchtes vllm/vllm-openaiWeights Repo Updated Sep 16, 2026

2× RTX 3090

Qwen3.8 27B AWQ with SGLang on 2× RTX 3090

113tok/sPeak

1 request, best value reported by source, context 8k Source

Intelligence34with thinking no thinking 20

License unclear

AWQ-INT4 W4A16 (Marlin), bf16 KVSGLang commit 2e7c85da6Weights Repo Updated Sep 23, 2026

4× RTX PRO 6000

MiniMax-M3 428B-A23B NVFP4 with SGLang on 4× RTX PRO 6000

87tok/sPeak

1 request, best value reported by source Source

Intelligence29with thinking

License unclear

NVFP4 (modelopt_fp4), BF16 KVSGLangWeights Repo Updated Sep 16, 2026

4× RTX PRO 6000

GLM-5.2 526B NVFP4 with vLLM on 4× RTX PRO 6000

77tok/sPeak

1 request, best value reported by source, context 131k Source

Intelligenceno independent value

Custom kernel required

NVFP4 routed experts, FP8 KVvLLMWeights Repo Updated Aug 16, 2026

4× RTX PRO 6000

Kimi-K2.6 519B NVFP4 with vLLM on 4× RTX PRO 6000

71tok/sPeak

1 request, code prompt Source

Intelligenceno independent value

License unclear

NVFP4, fp8_e4m3 KVvLLMWeights Repo Updated Sep 4, 2026

1× Strix Halo

Qwen3.6-35B-A3B Q4_K_M with llama.cpp on 1× Strix Halo

70.2tok/sPeak

1 request, synthetic prompt Source

Intelligence18with thinking no thinking 15

GGUF Q4_K_M; Variante DYNAMIC (Mixed per-tensor)llama.cppWeights Repo Updated May 30, 2026

4× RTX 3090

Ornith-1.5-35B-A3B EXL3 3 bpw with ExLlamaV3 on 4× RTX 3090

50tok/sPeak

1 request, best value reported by source Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality

EXL3 3 bpw Experten (MCG), Rest BF16, MTP 4 bpwexllamav3 (TabbyAPI) 1.4.3 (+ Patch tp_import_split_n)Weights Repo Updated Aug 26, 2026

1× RTX 3090

Qwen3.8 27B AWQ with SGLang on 1× RTX 3090

43.2tok/sPeak

1 request, best value reported by source, context 8k Source

Intelligence34with thinking no thinking 20

AWQ-INT4 W4A16 (Marlin), bf16 KVSGLang commit 2e7c85da6Weights Repo Updated Sep 23, 2026

8× RTX 3090

GLM-4.7 218B AutoRound INT4 with vLLM on 8× RTX 3090

38.5tok/sPeak

1 request, mixed prompt set Source

Intelligenceno independent value

W4A16 AutoRound (group 128) + FP8 KVvLLMWeights Repo Updated May 30, 2026

1× DGX Spark

DeepSeek-V4-Flash EXL3 3.0 bpw with vLLM on 1× DGX Spark (256k context)

38.1tok/sPeak

1 request, code prompt Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Experimental
Custom kernel required

REAP-pruned + EXL3/Trellis TR3 ~3.0 bpw experts, FP8 carried tensors; KV: 584-Byte padded FP8 sparse-MLA (nvfp4_ds_mla-Pfad)vLLMWeights Repo Updated Sep 25, 2026

1× DGX Spark

DeepSeek-V4-Flash 180B NVFP4 with vLLM on 1× DGX Spark

24.4tok/sPeak

1 request, best value reported by source Source

Intelligenceno independent value

MXFP4/NVFP4-style experts, FP8 MLA KVvLLMWeights Repo Updated Sep 17, 2026

4× DGX Spark

GLM-5.3-Flash 320B-A18B NVFP4 with SGLang on 4× DGX Spark

22.7tok/sPeak

1 request, best value reported by source Source

Intelligence42with thinking

NVFP4 weights + FP8 E4M3 KVSGLangWeights Repo Updated Sep 15, 2026

1× DGX Spark

GLM-5.3-Flash 320B-A18B EXL3 2.0 bpw with vLLM on 1× DGX Spark

18.9tok/sPeak

1 request, best value reported by source, context 4k Source

Intelligence42with thinkingOriginal model's value. Heavily compressed (~2 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 K2 2.0 bpw (codebook mcg) routed experts, fp8_ds_mla KVvLLMWeights Repo Updated Sep 25, 2026

2× DGX Spark

GLM-5.3-Flash 320B-A18B NVFP4 with SGLang on 2× DGX Spark

17.5tok/sPeak

1 request, best value reported by source Source

Intelligence42with thinking

NVFP4 weights + FP8 E4M3 KVSGLangWeights Repo Updated Sep 15, 2026

8× RTX 3090

Qwen3.5 35B-A3B with vLLM on 8× RTX 3090

13.9tok/sPeak

1 request, best value reported by source Source

Intelligence19with thinking no thinking 15

Experimental

vLLMWeights Repo Updated Sep 10, 2026

1× DGX Spark

DeepSeek-V4-Flash 180B IQ2_XXS with DwarfStar on 1× DGX Spark

13.6tok/sPeak

1 request, best value reported by source, context 2048 Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality

Q2-REAP-ds4 (IQ2_XXS gate/up, Q2_K down, Q8_0 shared/attn)DS4 (antirez/ds4)Weights Repo Updated May 30, 2026

1× DGX Spark

DeepSeek-V4-Flash 162B IQ2_XXS with DwarfStar on 1× DGX Spark

12.8tok/sPeak

1 request, best value reported by source, context 2048 Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality

Q2-REAP-ds4 (IQ2_XXS gate/up, Q2_K down, Q8_0 shared/attn)DS4 (antirez/ds4)Weights Repo Updated May 30, 2026

1× DGX Spark

GLM-5.3-Flash 320B-A18B EXL3 3.05 bpw with SGLang on 1× DGX Spark

10.8tok/sPeak

1 request, best value reported by source Source

Intelligence42with thinkingOriginal model's value. Heavily compressed (~2.3 bit), intelligence is likely much lower.

≤ 3 bit: quantization may cost quality

EXL3 mixed: 12 Layer 3.05 bpw + 33 Layer 2.05 bpw (mul1), fp8 KVSGLangWeights Repo Updated Sep 19, 2026

1× DGX Spark

Laguna-S-2.1 EXL3 3.25 bpw with vLLM on 1× DGX Spark

no documented speed

Intelligenceno independent value

Hybrid: 64 Hot-Experten NVFP4 + 192 Tail-Experten EXL3 Trellis-3 (nominal 3.25 bpw)vLLM (EXL3-Tail-Overlay)Weights Repo Updated Jul 26, 2026

← Back to overview