FR
Réserver un appel

Local AI · Recipe · 2× DGX Spark

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams)

GLM-5.3-Flash EXL3 with TensorFold on 2× DGX Spark (1M context, 4 streams): 97.6 tok/s according to github.com (dataset as of Sep 29, 2026).

by MiaAI-Lab

97.6tok/sEveryday

HumanEval 97.6% / GSM8K 98.0% with FP8 KV and 4-bit dense

Source: github.com
Intelligence (original model) · Artificial Analysis42with thinking

≤ 3 bit: quantization may cost quality

Instructions in the README Repo

Engine

Engine
TensorFold v0.5.0 (52 patches: DFlash2+copy drafts, 4-bit dense, FP8 KV, RoCE one-shot all-gather, shared KV pool, vision, tool calling)
Quantization
EXL3/TR3 4bpw routed experts + 4-bit dense, BF16 elsewhere, FP8 KV
Model family
GLM-5.3-Flash
Context
1,048,576
Parameters
320B MoE / 18B active
Creator
MiaAI-Lab
GitHub stars
21
Repo updated
Sep 30, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)60.4 tok/sProsa, 1 Request, GPU-Clocks 2200 MHz, sparkDash via OpenAI-API; TTFT 170 ms Source 

Decode · parallel requests

Sums over all streams are marked; per request it is correspondingly less.

  • 4 parallel requests228 tok/sPeak (decode) · Strukturierte Ausgabe ×4 aggregiert (61,1 pro Request); Prosa ×4: 108,8 aggregiert Source 

Prefill by context

  • 32K1,979 tok/s32.790-Token-Prompt (TTFT 16,57 s); 131k: 1.837,9; 262k: 1.641,7; 981k NiaH: 1.015, Needle gefunden Source 

Time to first token

Shorter is better.

  • Prompt not stated170 msProsa bei 1 Request (strukturiert: 149 ms) Source 

What you need

Hardware
2 × NVIDIA DGX Spark (GB10)
Weights
Engine
TensorFold v0.5.0 (52 patches: DFlash2+copy drafts, 4-bit dense, FP8 KV, RoCE one-shot all-gather, shared KV pool, vision, tool calling)
Context
1,048,576 tokens

Notes

What matters before you rebuild it.

  • ≤ 3 bit: quantization may cost qualityAt 3 bit and below the model may answer noticeably worse than the original. The intelligence number refers to the original.
  • Custom kernel required
  • vision
  • tool-calling
  • Non-commercial
  • License unclear

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview