IT
Prenoti una chiamata

Local AI · Recipe · 4× RTX 3090

Qwen3.8-Flash-Next 122B AWQ with vLLM on 4× RTX 3090

by alesha-pro

No speed value is documented for this recipe, so this page is not indexed.

Intelligence (original model) · Artificial Analysis40with thinking
Instructions in the README Repo

Engine

Engine
vLLM
Quantization
AWQ INT4 (cyankiwi, lokal abgeleitet) + externes FP8-PLE (RadixArk) + kalibrierter FP8-E4M3-QSA-KV
Model family
Qwen3.8-Flash-Next
Context
262,144
Parameters
122B total / 51B active (laut README)
Creator
alesha-pro
GitHub stars
15
Repo updated
Aug 30, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Time to first token

Shorter is better.

  • Prompt not stated1,754 mscontext 32K · Warm TTFT v1 (Prefix-Cache) bei 32,768 Input-Tokens (README: 1.754 s) – Replay, nicht als Prefill-Hauptwert verwenden; Quelle: „1.754 s“ (s→ms umgerechnet, nicht gerundet) Source 

What you need

Hardware
4 × NVIDIA RTX 3090
Engine
vLLM
Context
262,144 tokens

Notes

What matters before you rebuild it.

  • Experimental
  • Custom kernel required
  • License unclear

Sources

Related recipes

2× DGX Spark

Qwen3.8-Flash-Next 125B INT4-AutoRound with TensorFold on 2× DGX Spark (1M context)

by MiaAI-Lab

89.3tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligenceno independent value
  • Custom kernel required
  • License unclear

INT4-AutoRound (GPTQ int4, Top-5-Experten) + FP8 n-gram-Tabelle, FP8 KVTensorFold tensorfold-native (Zig, zig-flashnext db281878 + patches), Image ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold:zig-db28187-f3e1300f4f62, TP=2 über CX7/RoCE, MTP + Copy-Drafts (QUANT=int4ar)Weights Repo Updated Oct 26, 2026

Details
2× DGX Spark

Qwen3.8-Flash-Next INT4-AutoRound (v5.2) with vLLM on 2× DGX Spark

by myllmbox

74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

Con il vostro consenso utilizziamo Google Analytics (Google, USA) e la nostra misurazione dell’audience per capire come viene utilizzato il nostro sito. Le memorizzazioni strettamente necessarie, come la vostra scelta sui cookie, avvengono senza consenso. Potete modificare la vostra scelta in qualsiasi momento tramite «Impostazioni cookie» nel footer. Informativa sulla privacy · Informativa sui cookie