Local AI · Recipe · 4× RTX 3090

Qwen3.8-Flash-Next 122B AWQ with vLLM on 4× RTX 3090

by alesha-pro

No speed value is documented for this recipe, so this page is not indexed.

Intelligence (original model) · Artificial Analysis40with thinking

Instructions in the README Repo

Engine: vLLM
Quantization: AWQ INT4 (cyankiwi, lokal abgeleitet) + externes FP8-PLE (RadixArk) + kalibrierter FP8-E4M3-QSA-KV
Model family: Qwen3.8-Flash-Next
Context: 262,144
Parameters: 122B total / 51B active (laut README)
Creator: alesha-pro
GitHub stars: 15
Repo updated: Aug 30, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Time to first token

Shorter is better.

Prompt not stated1,754 mscontext 32K · Warm TTFT v1 (Prefix-Cache) bei 32,768 Input-Tokens (README: 1.754 s) – Replay, nicht als Prefill-Hauptwert verwenden; Quelle: „1.754 s“ (s→ms umgerechnet, nicht gerundet) Source

What you need

Hardware: 4 × NVIDIA RTX 3090
Weights: Main weightscyankiwi/Qwen3.8-Flash-Next-AWQ-INT4 other
pleRadixArk/Qwen3.8-Flash-Next-NVFP4 other
Engine: vLLM
Context: 262,144 tokens

Notes

What matters before you rebuild it.

Experimental
Custom kernel required
License unclear

Sources

Related recipes

1× Mac

Qwen3.8-Flash-Next 125B-A6B MLX 4/8-bit with mlx-serve on 1× Mac (M5 Ultra)

by drowzeys

155tok/sEveryday

1 request, llmprobe standard prompt, thinking on, with MTP + PLD (per recipe) Source

Intelligence40with thinking

Experimental
License unclear

MLX mixed 4/8-bit (ddalcu, 100 GiB), KV bf16mlx-serve 26.9.7-dev @933566c, MLX 0.32.3 (NAX an)Weights Repo Updated Sep 30, 2026

Details

4× RTX 3090

Qwen3.8-Flash-Next 125B with llama.cpp on 4× RTX 3090

by tonyd2wild

110tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source

Intelligence40with thinking

License unclear

W4A16 (vLLM) bzw. GGUFvLLM (Lane 1) / llama.cpp (Lane 2)Weights Repo Updated Oct 1, 2026

Details

2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source

Intelligenceno independent value

Custom kernel required
License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details

2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source

Intelligence40with thinking

Custom kernel required
License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details

4× DGX Spark

Qwen3.8-Flash-Next NVFP4 with SparkRing on 4× DGX Spark

by fujitsupolycom

73tok/sEveryday

1 request, decode at 32K context (speed table, ring of four), thinking on, xhigh Source

Intelligence40with thinking

Custom kernel required

NVFP4 QAD (Local Inference Lab)vLLM (local-inference-lab Fork) via SparkRing (SIRCL-Collectives, switchless RDMA ueber ConnectX-7), Image Release 2026.10.2Weights Repo Updated Nov 2, 2026

Details

2× DGX Spark

Qwen3.8-Flash-Next MLX 4-bit with TensorFold on 2× DGX Spark

by sfxnz

69.6tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source

Intelligence40with thinking

License unclear

MLX affine 4-bit (group 32) auf allen Linears inkl. n-gram-Tabellen und MTP-Head, KV bf16TensorFold v0.6.2 (56e2e3e), TP=2, Image tf-qwen38-flashnext:0.6.2 (nvcr.io/nvidia/pytorch:26.07-py3)Weights Repo Updated Oct 2, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshop Back to overview

Engine

Measurements

Time to first token

What you need

Notes

Sources

Related recipes

Qwen3.8-Flash-Next 125B-A6B MLX 4/8-bit with mlx-serve on 1× Mac (M5 Ultra)

Qwen3.8-Flash-Next 125B with llama.cpp on 4× RTX 3090

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

Qwen3.8-Flash-Next NVFP4 with SparkRing on 4× DGX Spark

Qwen3.8-Flash-Next MLX 4-bit with TensorFold on 2× DGX Spark

Local AI in your company?