EN
Book a call

Local AI · Recipe · 1× DGX Spark

Laguna-XS-2.1 33B NVFP4 with vLLM on 1× DGX Spark

Laguna-XS-2.1 33B NVFP4 with vLLM on 1× DGX Spark: 43.8 tok/s according to github.com (dataset as of Nov 3, 2026).

by r0b0tlab

43.8tok/sMixed

1 request, mixed prompt set, no speculative decoding

Source: github.com
Intelligence (original model) · Artificial Analysisno independent value
Instructions in the README Repo

Engine

Engine
vLLM v0.24.0
Quantization
NVFP4 (compressed-tensors) + FP8 KV
Model family
Laguna-XS-2.1
Context
32,768
Parameters
33B total / 3B active MoE
Creator
r0b0tlab
GitHub stars
0
Repo updated
Jul 4, 2026

Measurements

Every sourced value of this recipe, each with its condition and source. Bars relative to the largest value in the group.

Decode · 1 request

  • Mixed (decode)43.8 tok/scontext 512 in / 256 out (random) · Vllm bench serve, random, --ignore-eos, 20 Prompts; TTFT p50 120.5 ms Source 
  • Peak (decode)43.8 tok/scontext 512 in / 256 out (random) · Vllm bench serve, random, --ignore-eos, 20 Prompts; TTFT p50 120.5 ms Source 

Prefill by context

  • 2K8,975 tok/s--random-input-len 2048 --random-output-len 1; TTFT p50 228.2 ms Source 

Time to first token

Shorter is better.

  • Prompt not stated121 mscontext 512 in / 256 out (random) · TTFT p50 C=1 Source 

What you need

Hardware
1 × NVIDIA DGX Spark (GB10)
Weights
Engine
vLLM v0.24.0
Context
32,768 tokens

Sources

Related recipes

1× DGX Spark

Nemotron-3.5-Bolt 74B-A9B 24GB pack with TensorFold on 1× DGX Spark

by MiaAI-Lab

71.8tok/sEveryday

1 request, prose prompt (sampled, thinking off), with LiLiCorr (per recipe) Source 

Intelligenceno independent value
  • ≤ 3 bit: quantization may cost quality
  • Custom kernel required
  • License unclear

24GB pack: IQ2_XXS experts (2.0625 bits), NVFP4 Mamba projections, Q8_0 attention/latent projections/lm_head; bf16 KVTensorFold tensorfold-native v1.1.6 (eea9dc9c) + recipe patches/ (Bolt lane backend, libbolt.so CUDA forward pass), LiLiCorr draftsWeights Repo Updated Nov 1, 2026

Details
1× DGX Spark

MiniCPM-V-4.7-35B-A3B EXL3 6.08 bpw with ExLlamaV3 on 1× DGX Spark (text, 262k context)

by r0b0tlab

59.4tok/sEveryday

1 request, chat prompt (20 prompts, thinking off), no drafting (default profile) Source 

Intelligenceno independent value
  • Custom kernel required

EXL3 6.08 bpw (28,3 GB), FP16 KV; nur Textpfad (Vision-Tower wird von ExLlamaV3 nicht ausgeführt)ExLlamaV3 1.5.0 (Fork r0b0tlab/exllamav3-r0b0t, Branch minicpm-v4.7-dspark @ 5ddae7d12448), torch 2.14.0+cu130Weights Repo Updated Oct 14, 2026

Details
1× DGX Spark

Qwen3.8-Flash-Next NVFP4 (NVIDIA QAD, FP4-Local) with TensorFold on 1× DGX Spark

by MiaAI-Lab

55.6tok/sEveryday

1 request, prose prompt, with MTP (per recipe) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

NVIDIA FP4-QAD-Export (FP4-Local @b9ce18a6): Routed Experts NVFP4 (Gruppe 16), Attention/Shared Expert MXFP8, alle 512 Experten mit 10 pro Token; n-gram-Tabelle per pread von der SSD; FP8 KVTensorFold tensorfold-native (Zig) v1.1.4 (eab85828) + Patches 0001–0017, Image tensorfold-qwen38:v1.1.4-eab8582 (GHCR v1.1.4-eab8582-4baf16137ecd) auf nvcr.io/nvidia/pytorch:26.07-py3, MTP-DraftsWeights Repo Updated Oct 30, 2026

Details

← Back to overview

Local AI in your company?

In a workshop we work out which models and which hardware fit your tasks, and build the first agent on your infrastructure.

See the AI workshopBack to overview
Cookies

With your consent we use Google Analytics (Google, USA) and our own audience measurement to understand how our website is used. Strictly necessary storage, such as your cookie choice, happens without consent. You can change your choice at any time via “Cookie settings” in the footer. Privacy Policy · Cookie Policy