---
type: "Article"
title: "Nanbeige4.3-33B W4A16 AutoRound with vLLM on 2× RTX 3090"
description: "Nanbeige4.3-33B W4A16 AutoRound with vLLM on 2× RTX 3090: recipe by tonyd2wild with speed, condition and source for every number, weights and license."
resource: "https://www.contextstudios.ai/local-ai/tonyd2wild--nanbeige4.3-33b-w4a16-autoround-2x-rtx-3090"
language: "en"
generated:
  by: "process:contextstudios-md/1"
  at: "2026-11-11T01:16:33.633Z"
status: "stable"
---

# Nanbeige4.3-33B W4A16 AutoRound with vLLM on 2× RTX 3090

Nanbeige4.3-33B W4A16 AutoRound with vLLM on 2× RTX 3090: 62.2 tok/s according to github.com (dataset as of Nov 3, 2026).

- Creator: [tonyd2wild](https://www.contextstudios.ai/local-ai/creators/tonyd2wild.md)
- Engine: vLLM 0.27.1
- Quantization: W4A16 AutoRound (GPTQ INT4, g128) + FP8 e5m2 KV
- Model family: Nanbeige4.3-33B
- Hardware: rtx-3090 x2
- Context: 131072

## All numbers

- 62.2 tok/s (Everyday; Everyday, 1 request, prose prompt, no speculative decoding) — [source](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090#c1-to-c6-sweep-per-stream-decode-toks-total-toks)
- decode_tps: 62.2 (1 request) — [source](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090#c1-to-c6-sweep-per-stream-decode-toks-total-toks)
- decode_tps: 66.2 (1 request) — [source](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090#c1-to-c6-sweep-per-stream-decode-toks-total-toks)
- prefill_tps: 1765 (1 request) — [source](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090/blob/main/results/bench_c1c6_2026-10-25.md)
- ttft_ms: 60 (1 request) — [source](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090/blob/main/results/bench_c1c6_2026-10-25.md)

## Weights

- [Tech2wild/Nanbeige4.3-33B-W4A16-AutoRound](https://huggingface.co/Tech2wild/Nanbeige4.3-33B-W4A16-AutoRound) (main, apache-2.0)
- [Nanbeige/Nanbeige4.3-33B](https://huggingface.co/Nanbeige/Nanbeige4.3-33B) (base, apache-2.0)

[https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090)

## Related

- [Local AI on your hardware.](https://www.contextstudios.ai/local-ai.md)