A Trillion-Parameter Model on Your Own Hardware: What Mistral Large 4 Means for a 4× DGX Spark Cluster

tg-mistral-large-4-dgx-spark-cluster

A Trillion-Parameter Model on Your Own Hardware: What Mistral Large 4 Means for a 4× DGX Spark Cluster

The short answer: Mistral Large 4 is, at 1.05 trillion parameters with only 52 billion active per token, the largest openly announced model yet — and we answered the does-it-run-at-home question on our 4× DGX Spark cluster using the largest reference model available to us: 23.0 tokens per second single-stream, 42.2 on the 8-stream aggregate, and 0.553 € of electricity per million output tokens. The open weights are listed for October 31, 2026 on Hugging Face; the license is still unannounced.

On October 6, 2026, Mistral launched the public preview of Mistral Large 4 — internally the model goes by "le Chonk." It is the company's largest model: 1.05 trillion parameters in a fine-grained mixture-of-experts architecture, 52 billion active per token (49 billion pure activation plus 3 billion in embedding and output layers), plus a 1.6-billion-parameter vision encoder for images and documents. Context length is 1 million tokens. Mistral says it trained the model on 3,800 NVIDIA Grace Blackwell GPUs in its own European datacenters.

The announcement, however, is only half the story. The interesting question for anyone weighing own hardware: what does a model of this class mean for self-hosting — and what does a locally generated million tokens really cost? We measured our 4× DGX Spark cluster and ran the numbers against the API pricing.

The facts about Mistral Large 4 at a glance

One caveat: the open-weights promise is an announcement, not a fact. On Hugging Face the model sits as an "upcoming release" with an ETA of October 31, 2026, and Mistral had not named a license at press time. All model facts come from the official product page and the API docs; third-party benchmarks are reported as such.

Why the 1T class is different for self-hosting

This autumn's flash generation — GLM-5.3-Flash, DeepSeek V4.1 Flash, Qwen3.8-Flash-Next — runs on consumer hardware because it activates only 6 to 18 billion parameters per token. Mistral Large 4 activates 52 billion, roughly three times as many. Concretely:

• Memory is not the problem; speed is. A 4-bit quant of the 1.05T weights needs roughly 525 GiB plus KV cache — feasible on four Sparks with 512 GB of unified memory plus system RAM, not on a single 128 GB box.
• Bandwidth decides. At 52B active parameters in 4-bit, about 26 GB must be read per token. Even with 4×273 GB/s pooled, that is a different regime than the flash class's 6B activation — our measured 23 tok/s vs. 50+ for Flash-Next makes it tangible.
• Batching is the lever. The single-stream number sounds modest; the aggregate across parallel streams is the figure that matters for team and agent operation.

We did not theorize these three points — we measured them on the largest model already stable on our hardware: Qwen3.8-Flash-Next.

Our own measurement

Own run, October 20, 2026, 10:30 a.m. CEST. Endpoint: http://127.0.0.1:8212 (vLLM, OpenAI-compatible), model qwen3.8-flash-next-nvfp4. The endpoint also serves our production pipelines; these are operational, not laboratory numbers.

Two things stand out. First, the 0.389-second TTFT shows prefill is strong on the GB10 die — responses start promptly even while the stream itself runs at a moderate rate. Second, the jump from 23.0 to 42.2 tok/s on the 8-stream aggregate confirms the old rule that this hardware class lives on batching. For a single user watching the cursor, the 23 counts; for an agent server juggling eight jobs, the 42.2 does.

The cost math: 0.553 € per million tokens — and what it means

Our upper bound for energy cost: at an assumed 240 W nameplate (PSU spec, not measured) and an assumed commercial rate of 0.35 €/kWh, the delivered work costs 0.553 € per million output tokens — computed on the 42.2 tok/s 8-stream aggregate, hardware depreciation excluded.

Set against the prices from our own cost monitor:

The honest reading: against the cheapest flash APIs the local math loses on pure tokens — against Mistral Large 4's $2.09 output price it wins clearly, by a factor of ~3.8 ($2.09 ≈ €1.92 at 0.92 $/€; our €0.553). The math shifts further once data residency forbids the cloud, high concurrency is required, or spend peaks like our $268.59/day matter. Cost was never the main argument for own silicon — control is.

The routing factor: why the gateway layer co-decides

A second cost lever sits outside the model: routing. The gateway market grew from 4.5 to over 100 trillion routed tokens per week in 2026, and default routes demonstrably favor the cheapest endpoints — with providers silently capping context windows and output limits. For your own math that means: the price per million on paper and the quality actually delivered per million are two different things, and the gap is created in the router, not the model.

For self-hosting this shifts the calculus once more: running locally bypasses routing discipline, provider caps, and price volatility entirely. That is the point where "more expensive per token" and "more reliable per job" say the same thing.

What we would actually do

• Wait for October 31. Only the open weights plus a license show whether self-hosting is legally and technically sensible. Until then everything is preview arithmetic.
• Treat 4× cluster as the target size. A single 128 GB machine does not cut it for the 1T class; four Sparks with 512 GB pooled plus RAM offload are the realistic floor for comfortable quants.
• **Watch activation, not zeros. 52B active is the reason for the pace — anyone used to 6B activation must calibrate expectations: half the flash-class speed, but a clear token-cost advantage against the 1T API.
• Plan for batching. 42.2 tok/s aggregate vs. 23.0 single-stream: this cluster's value appears in parallel operation, not at a single desk.

Frequently asked questions

Does Mistral Large 4 run on a single DGX Spark? No, not meaningfully. The 1.05 trillion parameters exceed a single device's 128 GB in any practical quantization; even aggressive 2-bit stages leave little room for KV cache and the OS. A lone Spark can hold the weights only with severe compromises and SSD streaming, which hurts speed further. The realistic floor for the 1T class is a 4-node pool with 512 GB of distributed memory plus system RAM for the remainder.

How fast is our cluster compared with the flash class? Our qwen3.8-flash-next-nvfp4 reference run on October 20, 2026 delivered 23.0 tok/s single-stream and 42.2 tok/s as the 8-stream aggregate, with a 0.389-second TTFT. Flash-class models activating 6B parameters reach markedly higher single-stream numbers on the same hardware; the trade-off is the smaller capability envelope. Mistral Large 4 at 52B active will sit below that reference single-stream; the aggregate across streams remains the lever.

What does a million output tokens cost locally? By our upper-bound math, 0.553 € — 240 W assumed nameplate times 0.35 €/kWh assumed commercial tariff, divided by the delivered 42.2 tok/s. Both assumptions are documented and adjustable in our config; real draw will be lower unless the machines run flat-out. Depreciation and networking are excluded. Against the API's $2.09 output price that is a factor of almost four; against the cheapest flash APIs the cloud wins.

Is self-hosting the 1T class worth it at all? Not for the pure token argument — but for three concrete cases. First, when data residency forbids the cloud round-trip and data must stay in-house. Second, when agent farms run sustained high concurrency and API peak billing matters; our own peak was $268.59 on a single day. Third, when independence from routing decisions and provider caps is the goal. Occasional chatting is cheaper on a flash API.

When and under which license do the weights ship? Per the Hugging Face placeholder, October 31, 2026; Mistral had not named the exact license at press time. Until then the model is available only via the paid preview API, at promotional $0.68 input and $2.09 output per million tokens. Before any self-hosting decision, check license and commercial terms; we will update the math once the weights and their conditions are fixed.

What is known about quality? From the product page: Mistral positions Large 4 as state of the art among open models and cites its own blind comparison rounds with professional raters; we have not verified those details independently and report them as claimed. For the self-hosting decision, quality matters less than license, memory footprint, and your workload profile. Our advice: after October 31, test community benchmarks against your actual application before ordering hardware.

Sources

• Mistral — Introducing Mistral Large 4 (Oct 6, 2026): https://mistral.ai/news/mistral-large-4
• Mistral Docs — Mistral Large 4, specs and API pricing: https://docs.mistral.ai/models/mistral-large-4
• Hugging Face — mistralai/Mistral-Large-4-1T-A52B, upcoming release, ETA Oct 31, 2026: https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B
• Own benchmark run, 4× DGX Spark cluster (vLLM, qwen3.8-flash-next-nvfp4), Oct 20, 2026, 10:30 CEST: http://127.0.0.1:8212
• Own cost monitor (cost-throttle), API spend peak $268.59/day on Aug 11, 2026, $0.43/day 30-day average
• daily-intel run #196 (Oct 6, 2026): Mistral Large 4, score 8.0; OpenRouter growth 4.5→146T tokens/week; HN 13:15 UTC