Qwen 3.8 27B Hardware Guide: From the RTX 3090 to the DGX Spark

Qwen 3.8 27B Hardware Guide: From the RTX 3090 to the DGX Spark

On August 14, 2026, Alibaba's Qwen team released the weights of Qwen3.8-27B under Apache 2.0 on Hugging Face – making it the most exciting local model in the 27B class right now. A dense, natively multimodal vision-language model (text, image, video) with a native context of 262,144 tokens (extendable to 1M via YaRN), a built-in multi-token prediction head (MTP), and an Artificial Analysis score of 52 – putting it ahead of much larger frontier models on the Agentic Index. Developer Sero (@0xSero) summed it up on X: a model that runs on ~$3,000 of hardware is suddenly beating everything that was state of the art four months ago.

The key question for anyone who wants to self-host it: which hardware runs this thing – and how fast? The honest answer has two layers. The baseline benchmarks from release weekend (llama.cpp without tuning), and what the community has built on top in just five days. Depending on the platform, the difference is a factor of 2 to 8. We compiled both: Hardware Corner test series, Simon Willison's experiments, GitHub repos from Mia AI Lab, r0b0tlab and syv-ai, Hugging Face discussions, the DFlash 2 release by z-lab/Inco AI, and X posts from the first days.

⚡ As of August 19, 2026. This ecosystem moves daily – every number is a snapshot with a source attached.

The model in 60 seconds

• 27.8 billion parameters, dense – no mixture-of-experts. Every generated token passes through the entire network. That makes memory bandwidth the decisive factor – and speculative decoding the biggest lever, because it verifies multiple tokens per forward pass.
• Architecture: 64 layers, 48 of them Gated DeltaNet and 16 full attention, plus a trained MTP head shipped inside the checkpoint.
• Multimodal: image and video understanding built in (GGUF users additionally need the ~0.93 GB vision projector).
• Reasoning modes: xhigh (default!), medium, low. The default overthinks massively – Simon Willison needed 21 minutes for an SVG prompt with it, and just over 2 minutes with reasoning off. For local hardware: use low or medium.

Memory requirements: the real ticket to entry

The full BF16 checkpoint weighs ~56 GB. Only quantization turns it into a consumer model:

On top comes the KV cache: with a Q4 build, Hardware Corner measured 18 GB at 4k, 22 GB at 64k, 26 GB at 128k and 34 GB at 256k context. The community has already pushed those limits – with FP8 KV cache (half the footprint), quantized embeddings (2.6 GB saved, more below) and KV quantization in llama.cpp. Rule of thumb regardless:

24 GB VRAM = 64k context out of the box. 32 GB = 128k. The full 262k only fits on 24 GB with tuning – or with unified memory.

---

1. Classic desktop PC with GPU (RTX 3090 / 4090 / 5090)

Baseline: llama.cpp, Q4, MTP disabled (Hardware Corner)

That is what you get if you simply start llama-server. Solid – but far below what the hardware can do.

What the community made of it (as of Aug 19)

RTX 3090 – star of the week. The repo syv-ai/qwen38-27b-rtx3090 shows what a tuned vLLM stack achieves on a single 3090 – even power-limited to 250 W: ~114 t/s single-user at default sampling (118–124 t/s greedy) with 64k context, still 95–100 t/s at 150k context – and in batch mode ~1,000 t/s aggregate across 64 parallel requests. The ingredients: int8 tensor-core GEMMs, MTP drafts with a reduced draft vocabulary, a calibrated int4 LM head, and a trick that frees up 2.6 GB of VRAM: Qwen3.8 has untied embeddings, and the public W4A16 quants carry two unquantized 2.5 GB BF16 matrices – the repo requantizes both to int8 with no measurable quality loss.

RTX 5090. A Japanese developer (note.com) had the model in his daily setup roughly one hour after release and measures 148.3 t/s on code and 128.6 t/s on Japanese prose with a tuned speculative-decoding config. A Hugging Face discussion reports 98 t/s with the Q6 quant plus MTP, and the SGLang team 200+ t/s with NVFP4 + speculative decoding. The KGP Talkie test series (45 configurations) confirms the mechanism: --spec-type draft-mtp at draft depth 3 = a free 1.81x, lossless, because speculative tokens are verified.

The 24 GB Blackwell proof. On an RTX PRO 4000 (24 GB), the model runs at 50.4 t/s in production average – with the full 256k context, using pinned llama.cpp patches. Embedded MTP lifted throughput there from 21.2 to 59.5 t/s (2.81x); the custom build added another +22% over clean master.

New verdict: The 3090 is no longer just the budget pick – with the right stack it is a full serving node. The 4090 benefits from the same levers. The 5090 remains the fastest single GPU, but the gap shrinks once speculation runs everywhere.

---

2. Laptop with GPU: the RTX 5080 case (16 GB)

This stays uncomfortable, for one single reason: the RTX 5080 Laptop GPU has 16 GB of VRAM – and the smallest usable 4-bit file is 17.1 GB. Q4 does not fully fit on the card. Three realistic options:

1. UD-Q3_K_XL (13.4 GB) entirely in VRAM. Runs cleanly with 8–16k context and benefits from the card's ~896 GB/s. Solid measurements for exactly this combination are still missing; bandwidth arithmetic suggests decode rates in the league of the desktop 24 GB baseline – with the quality hit of a 3-bit quant.
2. Q4 with partial offload to system RAM. A test of patience: the reference test on an RTX 5070 Ti Laptop (12 GB) delivered 3.3–4.5 t/s decode and 20–27 t/s prefill. On 16 GB the offloaded share is smaller, but the pattern holds: as soon as weights live in system RAM, speed collapses into single digits.
3. NVFP4 (~16.5 GB weights) is also too tight on 16 GB once KV cache and runtime are added.

Laptop verdict: 16 GB is the awkward middle for this model. If you buy a laptop specifically for it, go for the RTX 5090 Laptop GPU (24 GB) – Q4/Q5 fit entirely in VRAM there, and the desktop tuning recipes (MTP, DFlash 2) carry over, throttled by power limits and cooling. On the 5080, the model remains an experiment, not a daily driver.

---

3. Mac mini (M4 / M4 Pro)

Apple Silicon flips the logic: instead of a hard VRAM limit there is unified memory, almost all of which is available to the GPU. The price is bandwidth.

Reference point for the lower bound: an M2 Pro with 16 GB managed a measured 7.1 t/s with the 2-bit quant – the same machine failed outright at 4-bit. Only the M4 Pro with 2.3x the bandwidth turns the Mac mini into a serious Qwen host. Important for all Macs: use MLX instead of GGUF (details in the MacBook section) – on M5 chips that yields roughly 40% more decode.

---

4. MacBook Pro (M4 Pro / M4 Max / M5 Pro / M5 Max)

Baseline: llama.cpp/LM Studio on M5 Max (128 GB)

Simon Willison confirms the ballpark from practice: 15–30 t/s in LM Studio – enough for chat, coding agents (tested with Pi) and vision tasks, but noticeably slower than hosted APIs.

The MLX jump

The MLX community followed up quickly: a measured MLX conversion reaches ~33 t/s at 4-bit on the M5 Max with only 15.5 GB peak memory (8-bit: ~18 t/s at 28.9 GB, BF16: ~10 t/s at 54 GB). On the M5 Pro, a deployment test documents +40% over GGUF: from ~28 to ~40 t/s – the M5 GPU's neural accelerators pay off directly here. Two caveats: the plain mlx-lm conversion is text-only (the vision encoder and MTP drafter are stripped); if you need images, use the mlx-vlm builds from mlx-community. And since Aug 18 there is speculative decoding on the Mac too: the oMLX build with DFlash 2 support runs, per Inco's demo, on the M5 Max with the 4-bit MLX model plus the DFlash 2 drafter – putting values well beyond 40 t/s within reach on Apple Silicon as well.

Placing the variants: A MacBook Pro with M4 Pro (24–48 GB) behaves like the Mac mini M4 Pro. M4 Max / M5 Max from 36–48 GB is the comfortable Apple laptop choice. The big advantage over any Windows laptop remains: 64–128 GB of unified memory allows Q8, vision and long contexts at the same time – silently, on battery.

---

5. Mac Studio (M4 Max / M3 Ultra)

The Mac Studio is the quiet workstation path. At 546 GB/s the M4 Max sits well above the MacBook level, and decode rates rise accordingly – toward RTX 3090 baseline territory, without fan noise. The same rule applies: use MLX builds, and with oMLX + DFlash 2 speculation joins in. The M3 Ultra nearly doubles the bandwidth; hard to justify economically for Qwen 3.8 27B alone, interesting if 70B+ models are meant to run on the same machine.

---

6. NVIDIA DGX Spark (GB10): from problem child to serving node

The Spark is the best example of how fast the picture flipped in five days.

Baseline: llama.cpp, Q4 (Hardware Corner, Dell Pro Max GB10)

With 273 GB/s of LPDDR5x bandwidth, the dense 27B is sluggish in stock llama.cpp – Simon Willison got 15–30 t/s in LM Studio and already gained +72% with --spec-type draft-mtp.

Community state, Aug 19: SGLang + NVFP4 + speculation

Mia AI Lab (MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark) turned the Spark into a real server: ~33–35 t/s single stream with MTP and ~195–210 t/s aggregate across 10 parallel streams – with up to 1M context via YaRN, and no speed difference between 256k and 1M. Plus tangible system fixes: the GB10 hides 10 fast and 10 slow CPU cores – pinning the server to the fast ones gained +5%, and a fix for a doubled memory reservation in the GDN state pool freed further headroom. The sister repo delivers the vLLM counterpart with Unsloth NVFP4, FP8 KV cache and MTP.

r0b0tlab (qwen38-27b-nvfp4-sm121-sglang) provides the matching, cleanly matched evals on the Spark (NVFP4, think-off): DFlash 2 as production winner at 28.4 t/s single stream and a concurrency ladder from 23.5 (c1) via 55.0 (c4) up to 92.1 t/s at c8 – the EAGLE/MTP variant even reaches 123.9 t/s at c8. On the real 200-task quality set, DFlash 2 averaged 47.5 t/s end-to-end, +40% over the previous DSpark setup, at identical GSM8K/HumanEval/IFEval scores.

New Spark verdict: no longer a speed cripple, but a quiet 128 GB serving node for small teams – and it remains unique in that 256k–1M contexts simply fit.

---

7. AMD Strix Halo (Ryzen AI Max+ 395)

Strix Halo (Radeon 8060S, up to 128 GB LPDDR5X, 256 GB/s) is the x86 answer to Apple's unified memory – with a very active tuning community:

The key takeaways:

• MTP is the single biggest lever on Strix Halo – up to factor 2.83. But: llama.cpp's default draft depth of 16 halves the throughput; the optimum is 3–4.
• Backend split: ROCm delivers better prefill from 16k on (~330 vs. ~267 t/s), Vulkan better decode.
• Prefill is the Achilles' heel: a 32k prompt takes about two minutes to first token.
• Beware record numbers: the 60 t/s figures are cold-start bests; n-gram speculation can additionally distort benchmarks because repeated prompts train the cache – one tester saw an apparent ~180 t/s where ~47 was real. Realistic is 11–24 t/s in actual chat – after tuning, that's MacBook level, with 128 GB of memory at a much lower price. A DFlash 2 path for ROCm is emerging (the llama.cpp PR is backend-agnostic); measured Strix numbers for it are still pending.

---

Speed tuning: the four levers that work everywhere

1. Turn reasoning_effort down. The xhigh default is unusable on consumer hardware. Measured: reasoning off = 10x faster to a visible answer, medium costs almost nothing. Even the free HF community endpoint has since switched its default to medium.
2. DFlash 2 (new, Aug 18) – the new top lever. The block-diffusion drafter from z-lab/Inco AI (z-lab/Qwen3.8-27B-DFlash2, mirror of incoai/…) is a separate 2B draft model (~4 GB) that drafts entire token blocks in parallel; a lightweight selector picks the coherent path, the target model verifies – output provably identical (rejection sampling). The model-card measurement on an H200 (SGLang, block 8): single stream from 69 t/s autoregressive to 184–236 t/s depending on task (2.67–3.43x) – the native MTP head manages 135–179 t/s there. Mean acceptance length: 4.80 tokens per verify vs. 4.28 for MTP. Available in SGLang (PR #35371, merged Aug 19), vLLM, llama.cpp (--spec-type draft-dflash) and via oMLX on Apple Silicon.
3. Enable MTP when no DFlash 2 drafter is at hand (--spec-type draft-mtp, draft depth 3–4): +72% on the Spark, +81% on the 5090, +183% on Strix Halo. The head already ships inside the GGUF (layer 64) – no second download needed.
4. Free up memory: quantize the KV cache (q8_0/q4_0, or FP8 in vLLM/SGLang – half the KV footprint) and, on W4A16 quants, requantize the two unquantized 2.5 GB embedding matrices to int8 (2.6 GB VRAM back, no measurable quality loss). Exactly this combination brings 150k–262k context to 24 GB cards.

Concurrency: from single seat to serving – now with real numbers

Because Qwen 3.8 27B is dense, every token reads all 27B parameters – single stream is bandwidth-limited. That is exactly why batching scales disproportionately well: the weight reads are amortized across the batch. The community has now measured it:

Three rules follow:

• Speculation loses value as load rises – but at different rates. On the 3090 the crossover to plain continuous batching sat at ~8 parallel users (syv-ai). The z-lab H200 measurement shows the same pattern precisely: at c32, MTP/DSpark drop to 0.77–1.13x (sometimes slower than autoregressive), while DFlash 2 still holds 1.16–1.45x – a better drafter pushes the crossover far out.
• KV budget determines concurrency: ~64 KB/token BF16 per sequence in the 16 full-attention layers (FP8: half) – on a 5090 after NVFP4 weights, enough for 20+ parallel 8k sessions. SGLang pitfall on this hybrid GDN model: --mamba-full-memory-ratio (default 0.9) over-provisions the KV pool and silently caps concurrency.
• Ollama/LM Studio process requests largely sequentially – the wrong choice for serving. On Mac and Strix Halo, 2–4 parallel streams remain the sensible maximum.

Overall comparison (as of Aug 19, 2026)

Conclusion

Qwen 3.8 27B is the first open model to pack vision, 262k context, tool calling and serious coding ability into a 17 GB file. The most important lesson of the first week: the hardware is rarely the bottleneck – the inference stack is. Between stock llama.cpp and a tuned vLLM/SGLang setup with MTP or DFlash 2 lies a factor of 2 to 8, on identical hardware.

• Serving & value: a used RTX 3090 + the syv-ai stack – 114 t/s single, ~1,000 t/s in batch, for a fraction of a datacenter setup.
• Maximum single-user speed: RTX 5090 with NVFP4 + speculation (148–200+ t/s).
• Mac: Mac mini M4 Pro 48 GB to start, Mac Studio M4 Max 64 GB for comfort – in any case MLX over GGUF, and keep oMLX + DFlash 2 on your radar.
• On the go: MacBook Pro M5 Pro/Max from 48 GB (~40 t/s with MLX) or the RTX 5090 Laptop – not the 5080.
• Long context & small teams: DGX Spark with the Mia AI Lab or r0b0tlab recipe – 210 t/s aggregate at up to 1M context.
• 128 GB on a budget: Strix Halo, at MacBook level after MTP tuning.

And wherever you run it: set reasoning_effort to low/medium, enable DFlash 2 or MTP, quantize the KV cache and embeddings. These moves matter more than the next GPU generation.

---

Sources (accessed Aug 15–19, 2026): Qwen3.8-27B model card · Hardware Corner benchmarks · Simon Willison · syv-ai/qwen38-27b-rtx3090 (GitHub) · MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark (GitHub) & @MiaAI_lab on X · r0b0tlab/qwen38-27b-nvfp4-sm121-sglang (GitHub) · Inco AI: DFlash 2 & z-lab/Qwen3.8-27B-DFlash2 (H200 benchmarks) · RTX 5090 speculative setup (note.com) · 50 t/s on 24 GB Blackwell at 256k (piszczek.pl) · KGP Talkie: 45 llama.cpp configurations · @witcheer, @victormustar & @0xSero on X · MLX measurements M5 Max & mlx-community · Unsloth GGUF repo & guide · AMD day-0 blog · julianmb/q38rocm & KyaniteLabs (GitHub) · SGLang cookbook · Kingy.ai hardware guide · VentureBeat. All values are community or vendor measurements from the first five days after release; software states change daily.