Local AI · Recipe · 1× Mac

Qwen3.8-Flash-Next 125B with MLX on 1× Mac

Qwen3.8-Flash-Next 125B with MLX on 1× Mac: 83.1 tok/s according to github.com (dataset as of Nov 3, 2026).

by Weschera

83.1tok/sPeak

1 request, code prompt

Source: github.com
Intelligence (original model) · Artificial Analysis40with thinking

Engine

Engine
oMLX 0.6.4 (custom kernels)
Quantization
oQ4e gemischt (4-bit gesamt, group 64, MLX safetensors)
Model family
Qwen3.8-Flash-Next
Context
—
Parameters
125B MoE / 6B aktiv (+51B PLE, +4B MTP) lt. Schwester-Repo Qwen3.8-Flash-Next-1x-DGX-Spark; oMLX-Footprint ~103,94 GB
Creator
Weschera
GitHub stars
13
Repo updated
Aug 30, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Peak (decode)83.1 tok/scontext 5,999-token-Sequenz · 1.200-Zeilen-Sequenz, MTP6 + aggressive Burst; byte-identisch validiert; frischer Zustand ( kontrollierte Messung, kein Dauerzustand) Source 

Prefill by context

  • 15,416-token-Prompt215 tok/s215–220 prompt tok/s in Wiederholungsmessungen (Bereich); TTFT 70–72 s; balanced prefill nötig, damit der 15,416-token-Prompt admitted wird → approx Source 

What you need

Hardware
1 × Mac / Apple Silicon
Weights
Engine
oMLX 0.6.4 (custom kernels)

Related recipes

2× DGX Spark
89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark
74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.