EN
Book a call

Local AI · Recipe · 4× RTX 3090

Qwen3.8 27B NVFP4 with vLLM on 4× RTX 3090

Qwen3.8 27B NVFP4 with vLLM on 4× RTX 3090: 76.7 tok/s according to github.com (dataset as of Nov 3, 2026).

by alesha-pro

76.7tok/sMixed

1 request, mixed prompt set, context 8k, with MTP

Source: github.com
Intelligence (original model) · Artificial Analysis34with thinkingno thinking 20
Instructions in the README Repo

Engine

Engine
vLLM 0.26.0
Quantization
NVFP4 (compressed-tensors, Marlin-Kernel auf SM86) + FP8 KV
Model family
Qwen3.8
Context
262,144
Parameters
27B dense
Creator
alesha-pro
GitHub stars
13
Repo updated
Sep 28, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Mixed (decode)76.7 tok/scontext 8K · TP4/FP8 KV/no-MTP, median steady decode nach 1. Token, 128 generierte Tokens, temp 0.7, Prefix-Cache aus; synthetischer Prompt der Tiefe 8K Source 
  • Peak (decode)84.3 tok/scontext 254K · TP4/FP8 KV/MTP2 – vom Autor per verify-headline.log als Kurzfenster-Ausreißer entwertet (lange Generierungen ~75, wie ohne MTP); nur als Kontext, nicht als Hauptzahl Source 

Prefill by context

  • 8K824 tok/sTP4/FP8 KV/no-MTP, effektive Prompt-Tokens/s bis 1. Token (cold, Prefix-Cache aus) Source 

What you need

Hardware
4 × NVIDIA RTX 3090
Weights
Engine
vLLM 0.26.0
Context
262,144 tokens

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy