EN
Book a call

Local AI · Recipe · 2× RTX 3090

Qwen-AgentWorld 35B AWQ with vLLM on 2× RTX 3090

Qwen-AgentWorld 35B AWQ with vLLM on 2× RTX 3090: 144 tok/s according to github.com (dataset as of Nov 3, 2026).

by noonghunna

144tok/sEveryday

1 request, prose prompt, no speculative decoding

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value
Instructions in the README Repo

Engine

Engine
vLLM
Quantization
AWQ INT4 g32 asym compressed-tensors (cyankiwi), fp8_e4m3 KV
Model family
Qwen-AgentWorld
Context
262,144
Parameters
35B MoE / 3B aktiv
Creator
noonghunna
GitHub stars
2,471
Repo updated
Nov 2, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)144 tok/sKanonischer bench.sh: Narrativ = 800-Wort-Essay (max_tokens 1000), Code = Quicksort (max_tokens 800), T=0.6, 3 Warmups + 5 Läufe, mean wall TPS; wall, decode 147.12, n=5, CV ≤0.9%, TTFT 133 ms Source 
  • Peak (decode)144 tok/sKanonischer bench.sh: Narrativ = 800-Wort-Essay (max_tokens 1000), Code = Quicksort (max_tokens 800), T=0.6, 3 Warmups + 5 Läufe, mean wall TPS; wall, decode 147.12, n=5, CV ≤0.9%, TTFT 133 ms Source 

Prefill by context

What you need

Hardware
2 × NVIDIA RTX 3090
Engine
vLLM
Context
262,144 tokens

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy