IT
Prenoti una chiamata

Local AI · Recipe · 1× RTX 3090

Qwen3.8 27.3B with llama.cpp on 1× RTX 3090

Qwen3.8 27.3B with llama.cpp on 1× RTX 3090: 57 tok/s according to github.com (dataset as of Sep 29, 2026).

by alesha-pro

57tok/sEveryday

1 request, prose prompt, with MTP

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value
Instructions in the README Repo

Engine

Engine
llama.cpp
Quantization
Mirai-S-Trellis 2.4-bit (GGUF-Typen MS_V4T8/MS_V2T4/MS_V2T6/MS_I3, ids 90–93, nicht requantisiert) + q4_0- oder q8_0-KV
Model family
Qwen3.8
Context
131,072
Parameters
27.3B dense (Mirai S: 2.4 bits/weight, 8.45 GB)
Creator
alesha-pro
GitHub stars
10
Repo updated
Oct 7, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Everyday (decode)57 tok/scontext 6.9K-Prompt (prose) · Wie Zeile zuvor, Prosa-Anteil derselben MTP-Zelle Source 
  • Peak (decode)85 tok/scontext 6.9K-Prompt (code) · 128K ctx, q8_0 KV + MTP 3 (--spec-type draft-mtp --spec-draft-n-max 3), 14.6 GB peak; 85 code / 57 prose; 300 W Source 

Prefill by context

  • 2K1,225 tok/sChunked gated-delta-rule-Prefill: 2048 Tokens 1103 → 1225 tok/s gegen Off-Switch Source 

What you need

Hardware
1 × NVIDIA RTX 3090
Engine
llama.cpp
Context
131,072 tokens

Notes

What matters before you rebuild it.

  • Custom kernel required

Sources

Related recipes

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview