Local AI on your hardware.

From the first Spark to a cluster

A Spark has 128 GB of unified memory,¹ of which about 121 GiB are usable according to a recipe author.² That suits mid-sized MoE models: in MiaAI Lab's recipe, Qwen3.6-35B does 95.1 tok/s for a single request and 317 tok/s in total across eight.³

Two Sparks connect with a single QSFP cable.⁴ Three form a switchless ring with three cables.⁵ For four, NVIDIA specifies a switch with at least four 200 Gbit/s ports.⁶ Community projects such as SparkRing also run four boxes as a switchless ring; NVIDIA does not document that path.⁷

More boxes mainly add memory. DeepSeek-V4.1-Flash takes 476 GiB on disk and only fits from three Sparks up; on four, the recipe reports 87.7 tok/s for a single prose request.⁸

Tensor parallelism scales speed well but not linearly: in NVIDIA's test (Llama 3.3 70B, 32K input), decode on two Sparks is 2.0× and on four 3.7× as fast as on one.⁹

Why the Spark works despite 273 GB/s

273 GB/s of memory bandwidth is modest for a machine in this class.¹ During decode the model reads its weights again for every token. Three techniques make up for that.

MoE models activate only a fraction of their parameters per token. Speculative decoding (MTP, DSpark, DFlash) lets a small helper model propose several tokens that the large model checks in one step. Tensor parallelism splits every weight matrix across Sparks: in NVIDIA's test, time per output token drops from 269 ms (one Spark) to 133 ms (two) and 72 ms (four).⁹

Prefill of long prompts gains less: 1.6× and 2.1× at 32K tokens (computed from NVIDIA's TTFT values). NVIDIA itself notes that inference scales sub-linearly when nodes have to synchronise often.⁹

What to check when buying

Partner machines use the same GB10 chip (compute capability 12.1) as NVIDIA's Founders Edition, so the recipes are not tied to one model.¹⁰ The recipe's hardware note says which machine the author used.

The Founders Edition ships with 4 TB of NVMe.¹ That fills up quickly: Qwen3.8-Flash-Next alone needs about 130 GiB.²

For stacking, NVIDIA qualifies the Amphenol NJAAKK-N911 (0.4 m) or NJAAKK0006 (0.5 m) and Luxshare LMTQF022-SD-R cables. Faster cables do not help, since each port tops out at 200 Gbit/s.¹¹

GB10 machines differ: a software update cut the Founders Edition's idle draw clearly, while a Dell Pro Max GB10 stayed at 35–37 W.¹²

Power and noise

At idle, a Spark on current software draws 22–25 W.¹² LLM inference mostly takes 60–90 W, full load just under 200 W; outside stress tests it stayed below 40 dBA.¹³

At the average German household rate of 37.0 ct/kWh, a Spark running around the clock costs roughly €16–24 a month under typical inference load and about €6 at idle (our calculation, 720 hours).¹⁴