Local AI · Recipe · 2× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams): 60.4 tok/s according to github.com (dataset as of Nov 3, 2026).

by MiaAI-Lab

60.4tok/sEveryday

Prose decode, 1 request; quality (FP8 KV, 4-bit dense): GSM8K 98.8% / HumanEval 95.7% / HumanEval+ & MBPP+ 86.5%

Source: github.com
Intelligence (original model) · Artificial Analysis42with thinking

≤ 3 bit: quantization may cost quality

Engine

Engine
TensorFold v0.6.0 + Recipe v1.10 (README: 96 Patches; gepinntes Image v0.6.0-e66c657a433d mit 107 Patches: DFlash2+Copy-Drafts, 4-bit dense, FP8-KV, RoCE-One-Shot-All-Gather, Shared KV-Pool, optionaler Spill-Tier, Vision, Tool-Calling, Anthropic /v1/messages; PARALLEL=8 opt-in)
Quantization
EXL3/TR3 4bpw routed experts + 4-bit dense, BF16 elsewhere, FP8 KV
Model family
GLM-5.3-Flash
Context
1,048,576
Parameters
320B MoE / 18B active
Creator
MiaAI-Lab
GitHub stars
515
Repo updated
Oct 25, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)60.4 tok/sProsa, 1 Request, GPU-Clocks 2200 MHz, sparkDash via OpenAI-API; TTFT 170 ms Source 

Decode · parallel requests

Sums over all streams are marked; per request it is correspondingly less.

  • 4 parallel requests228 tok/sPeak (decode) · Strukturierte Ausgabe ×4 aggregiert (61,1 pro Request); Prosa ×4: 108,8. Opt-in PARALLEL=8 (v1.5): Prosa 130,8 / Code 167,0 aggregiert bei ×8 Source 

Prefill by context

  • 32K1,979 tok/s32.790-Token-Prompt (TTFT 16,57 s); 131k: 1.837,9; 262k: 1.641,7; 981k NiaH: 1.015, Needle gefunden Source 

Time to first token

Shorter is better.

  • Prompt not stated170 msProsa bei 1 Request (strukturiert: 149 ms) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Weights
Engine
TensorFold v0.6.0 + Recipe v1.10 (README: 96 Patches; gepinntes Image v0.6.0-e66c657a433d mit 107 Patches: DFlash2+Copy-Drafts, 4-bit dense, FP8-KV, RoCE-One-Shot-All-Gather, Shared KV-Pool, optionaler Spill-Tier, Vision, Tool-Calling, Anthropic /v1/messages; PARALLEL=8 opt-in)
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • vision
  • tool-calling
  • Non-commercial
  • License unclear

Sources

Related recipes

4× DGX Spark
68tok/sEveryday

1 request, generation at 32K context (speed table; measured only as a path of four Sparks inside a ring of eight, not as a ring of four), thinking always on (measured before the profile default high, i.e. template default max), with MTP3 (per recipe) Source 

Intelligence42with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details
2× DGX Spark
38tok/sEveryday

1 request, generation at 32K context (speed table, pair), thinking always on (measured before the profile default high, i.e. template default max), with MTP3 (per recipe) Source 

Intelligence42with thinking
  • Custom kernel required
  • License unclear

NVFP4/MXFP8 CSF QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.