Local AI · Creator

bertholomus

9 published recipes.

GitHub Hugging Face

Recipes

4× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 4.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

70.9tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 4.0 bpw (4,02 bpw, ~275 GB; eigener Quant)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp4 @0a0b69a, recipe v0.3.5) + kernel kit (362 files), TP4 über RoCE, DSpark drafts, CUDA graphsWeights Repo Updated Oct 24, 2026

Details

2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.5 bpw with TensorFold on 2× DGX Spark

by bertholomus

60.5tok/sEveryday

1 request, prose prompt (greedy, thinking off), with DSpark (per recipe) Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 2.5 bpw routed experts (2,52 bpw über die quantisierten Gewichte, ~185 GB; Attention/Shared Experts 4 bpw, DSpark-Drafter 4/6 bpw, Vision-Tower 16 bit)TensorFold native (Zig, TensorFold 1.0.12), BertholomusAI dsv41 family (bertholomus/TensorFold deepseek-v41-zig-tp2, recipe v0.7.3, served engine 6579b756dcd7) + kernel kit v0.7.0, TP=2 über RoCE, DSpark draftsWeights Repo Updated Oct 24, 2026

Details

2× DGX Spark

DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw (LibertAI) with TensorFold on 2× DGX Spark

by bertholomus

59.2tok/sEveryday

1 request, prose prompt (temperature 0, thinking off, single run), with DSpark (per recipe) Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 2.9 bpw (LibertAI, Revision afc7aa0a)TensorFold (Elevatormusic/TensorFold @07aa02585f64: TensorFold 0.6.6 line + BertholomusAI DeepSeek-V4.1 family), Launcher Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark, TP=2 über ConnectX-7, DSpark (MTP_DRAFTS=5), MEM_FLOOR_GIB=0Weights Repo Updated Oct 19, 2026

Details

4× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 4× DGX Spark

by bertholomus

53.5tok/sEveryday

1 request, prose prompt (temperature 0, single run), with DSpark (per recipe) Source

Intelligence34with thinking

Custom kernel required

offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mlavLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1, Image-ID sha256:3430d661…), DSpark K5, FlashInfer B12X MoE, CUDA Graphs, TP4 über Dual-Rail-RoCE v2, 8 Sequenzen / 16.384 Batched-TokensWeights Repo Updated Aug 13, 2026

Details

2× DGX Spark

DeepSeek-V4-Flash-0731 (official checkpoint) with vLLM + DSpark on 2× DGX Spark

by bertholomus

40.1tok/sEveryday

1 request, prose prompt (temperature 0), with DSpark (per recipe) Source

Intelligence34with thinking

Custom kernel required

offizieller Checkpoint (Revision 9e165c30), KV-Cache nvfp4_ds_mlavLLM (Image ghcr.io/anemll/dspark-vllm-gx10:0.1.1), DSpark K5, FlashInfer B12X MoE, CUDA Graphs [1, 2, 4, 8], TP2 über RoCE v2Weights Repo Updated Aug 4, 2026

Details

2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with TensorFold on 2× DGX Spark

by bertholomus

30.6tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe) Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)TensorFold (bertholomus fork, branch beam-1.0-tp2 @ d00201b4), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over 200 Gb/s RoCE, DFlash2 drafter (5 draft tokens, adaptive 3-5)Weights Repo Updated Oct 29, 2026

Details

2× DGX Spark

Beam-1.0-501B-A23B EXL3 3 bpw with vLLM on 2× DGX Spark

by bertholomus

28.3tok/sEveryday

1 request, prose prompt (~2K), with DFlash2 (per recipe) Source

Intelligenceno independent value

≤ 3 bit: quantization may cost quality
Custom kernel required

EXL3 3 bpw (routed experts 3 bit, attention/dense MLPs 6 bit, embedding/norms/router/lm_head BF16; ~90 GiB per node)vLLM TP2 (image beam-exl3-gb10 via gb10/build-image.sh: Reflection arm64 Beam vLLM image + vllm-exl3 d3cfd394 with Diffbot Beam patch, ExLlamaV3 5be88657 for sm_121, kernels xmoe_dense/xmoe_do8/x8rh/f8rh), DFlash2 drafter (3 draft tokens)Weights Repo Updated Oct 29, 2026

Details

2× DGX Spark

Ling-3.1-Flash-1019 EXL3 3.03 bpw with TensorFold on 2× DGX Spark

by bertholomus

~38.2tok/sPeakapproximate · method not stated

Best value reported by source, 1 request, prompt type not stated, 2,222 prompt tokens, range 37.7–38.7 tok/s, with MTP (per recipe) Source

Intelligenceno independent value

Custom kernel required

EXL3 3.03 bpw (routed experts 3, shared experts 5, KDA attention 4, dense MLP 4, MLA attention 6, head 6, MTP layer 4)TensorFold (bertholomus fork, branch ling-3.1-tp2 @ 72f16e48, bailing_v3 family on TensorFold v0.6.6 CUDA engine), image nvcr.io/nvidia/pytorch:26.07-py3, TP2 over direct RoCE, MTP drafting (adaptive depth)Weights Repo Updated Oct 29, 2026

Details

4× DGX Spark

GLM-5.3 EXL3 3.0 bpw with TensorFold on 4× DGX Spark

by bertholomus

~36tok/sPeakapproximate · method not stated

1 request, prompt type not stated, range 33–39 tok/s, with MTP-3 (per recipe) Source

Intelligence45with thinking

≤ 3 bit: quantization may cost quality
Custom kernel required
License unclear

EXL3 3.0 bpw (exllamav3 1.5.3; routed experts 3.0, attention/shared/dense ~5, MTP experts 4, lm_head 6; 3.04 bpw avg), Q5 latent cacheTensorFold 0.6.0 + bertholomus glm-dsa-tp4 branch (release numbers on 3eb35dd, head ddfad35), image nvcr.io/nvidia/pytorch:26.07-py3, TP4 over direct RoCE, MTP-3 drafting, config B (1M window, DCP4, --parallel 4)Weights Repo Updated Oct 9, 2026

Details

← Back to overview