Local AI · Creator

myllmbox

6 published recipes.

Recipes

2× DGX Spark
74tok/sEveryday

1 request, prose prompt (long story), thinking off, with MTP Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

INT4-AutoRound (azampatti): int4-Experten, fp8-Seitenschichten; n-gram-Tabelle beim Laden fp8 → NVFP4, halb pro Box auf der GPU; KV-Pin 29G/Box (1.829.182 Tokens)vLLM 0.30 (Image myllmbox/qwen38-flash-next-cluster-vllm:v5.2; RecoverSSM, dynamischer Draft-Depth 3–7 in Full-CUDA-Graphs, sampled Drafts + Block-Verifikation, One-Shot-RoCE-All-Reduce, Dual-Rail, TP=2, 64 Seats) — Kit v5.2Weights Repo Updated Oct 29, 2026

Details
2× DGX Spark
34.4tok/sEveryday

1 request, prose prompt (7,000-word story), with MTP (per recipe) Source 

Intelligence42with thinking
  • Custom kernel required
  • License unclear

NVIDIA FP4-Local-Checkpoint wie ausgeliefert (NVFP4-Experten, MXFP8-Dense-Layer); KV-Pool 1.145.238 Token (12G/Box); Marlin MoEvLLM v0.32.0 (Image myllmbox/glm53-flash-cluster-vllm:v3: mbx-Loader, NVFP4-Experten auf Marlin, dynamische MTP-Draft-Tiefe 1–9, One-Shot-RoCE-All-Reduce via b12x), TP=2 über RDMAWeights Repo Updated Oct 29, 2026

Details
1× DGX Spark
76.2tok/sPeak

1 request, code prompt (TypeScript), thinking off, v5.1 column (unchanged on the v5.2 image) Source 

Intelligence40with thinking
  • Custom kernel required
  • License unclear

hibrid48: kalibrierter Body mit NVFP4-Output-Head (0.33 GiB), bf16-KV (27G-Pin), NVFP4-n-gram-Tabelle, Marlin MoEvLLM 0.30 (RecoverSSM, dynamischer Draft-Depth bis 7, sampled Drafts mit Block-Verifikation, 16 Seats) — Kit v5.2; hibrid48 bleibt als Alternative (~20 % langsamer als INT4-Default)Weights Repo Updated Oct 30, 2026

Details
2× DGX Spark
49tok/sPeak

1 request, code-heavy spec prompt (SQL/TypeScript), thinking off, with MTP K=3 Source 

Intelligenceno independent value
  • Custom kernel required

Offizieller Checkpoint wie veröffentlicht (config.json: quant_method fp8, store_dtype mxfp4); KV-Pool 16 GB/Box ≈ 1,22M Token; Marlin MoEvLLM 0.30.1rc1 (Nightly ddd6fbca, Image myllmbox/mimo-v26-flash-cluster-vllm:v2: mbx-Loader, Split-KV-Attention beim Spec-Decoding), TP=2 über RDMA, MTP K=3 (3 trainierte Draft-Layer)Weights Repo Updated Oct 29, 2026

Details

← Back to overview