Cluster Two Mac Studios or Buy 512 GB? Local LLMs on the M5 Ultra, Fact-Checked

Short answer first: Yes. Since macOS 26.2, two Mac Studios with 256 GB each can be linked over Thunderbolt 5 with RDMA and jointly host a model that would not fit on one. But as of late October 2026, a cluster is mainly a tool for more capacity, not more speed. For the most interesting open model right now, DeepSeek V4.1 Flash 1017, a single M5 Ultra with 256 GB is enough. In the developers' measurements it generates around 90 to 100 tokens per second and reads long prompts at 3,000 to almost 3,900 tokens per second. The cluster measurements published so far for this model range from 16 to 36 tokens per second, and there is no published measurement yet for two M5 Ultras with working RDMA. Our recommendation: buy one 256 GB machine and add the second only when a model actually needs it. The 512 GB configuration is now mostly an insurance policy.

Status of this analysis: October 31, 2026. The engine landscape for Apple Silicon is changing week by week. For every number we name the hardware, model package and source, and we clearly separate measured, reported and estimated values.

The technology: RDMA over Thunderbolt 5

Until the end of 2025, clustering Macs was more of an experiment. The link ran over TCP, and every hop between two machines cost around 300 microseconds of latency. For methods in which the machines exchange data dozens of times per token, that was too slow.

With macOS 26.2, Apple introduced an RDMA driver for Thunderbolt 5. RDMA stands for Remote Direct Memory Access: one machine can write directly into the other's memory without going through the network stack. According to Jeff Geerling, latency drops below 50 microseconds, with practical throughput of 50 to 60 Gbit/s. Apple's machine learning framework MLX uses the driver from version 0.30.1 onward via the jaccl communication backend.

Two ways to split a model

• Pipeline parallelism: The model is split by depth. Mac 1 computes the first layers, Mac 2 the rest. Only a small intermediate result travels between the machines per token. This is robust, even over slow links, but brings practically no speed when generating, because the layers still run one after another. It only helps when reading very long prompts, because chunks can overlap. Apple measures up to 1.8× at 64,000 tokens.
• Tensor parallelism: Every single layer is split across both machines by width. Both compute at the same time but have to synchronize per token and layer. Without RDMA, communication eats the gain. With RDMA, this is the path to real speedups.

What Apple itself measures

On October 15, 2026, Apple published its own measurements on the M5 Ultra. Decode speed in tokens per second:

The reading is clear: pipeline brings capacity, tensor parallelism brings capacity and speed, roughly a factor of 1.6 to 1.8 with two machines. This distinction later decides whether a cluster is worth it.

What you need for the setup

• RDMA has to be enabled once on each Mac in Recovery mode with rdma_ctl enable.
• The Macs are connected directly with Thunderbolt 5 cables, and with more than two machines each one to every other. The Thunderbolt 5 port next to the Ethernet port is not suitable. In its tests, Apple uses two cables per link to double the bandwidth.
• All machines need exactly the same macOS version, so updates must always be installed together.
• A common pitfall in practice: if the Thunderbolt link only has a self-assigned 169.254 address, the connection silently falls back to LAN or even Wi-Fi. The fix is a manual IPv4 address on the Thunderbolt port of both Macs, for example 10.10.0.1 and 10.10.0.2.

The memory math: what fits in 2 × 256 GB?

A 256 GB Mac does not give the GPU the full 256 GB. macOS needs memory itself, and the GPU limit has to be raised with sudo sysctl iogpu.wired_limit_mb. One user runs two headless M5 Ultras with 250 GB set manually per machine. Realistically, a cluster leaves around 450 to 500 GB for weights, KV cache and working buffers. This is our estimate, not a vendor figure.

The rule of thumb for the weights: parameters times bits per weight divided by eight.

* Including the scaling factors of the NVFP4 format, Apple puts Muse Spark Open at around 493 GB.

The result: at 3 to 3.5 bits per weight, the entire class of models with 750 to 820 billion parameters fits into two 256 GB machines. Models with more than one trillion parameters do not. For the four-bit version of Muse Spark Open, Meta specifies two Mac Studios with 512 GB each.

Case study: DeepSeek V4.1 Flash 1017

On October 17, 2026, DeepSeek released an update to its Flash model, with open weights under the MIT license. The model is special for local hardware for several reasons:

• Architecture: It is the first DeepSeek model with a causal encoder-decoder architecture. 40 layers are split into 20 encoder and 20 decoder layers. The backbone has 552 billion parameters, of which only 8 billion are active per token when reading and 16 billion when generating.
• Tiny KV cache: According to the model card, the KV cache needs only 890 bytes per token. Even a million tokens of context takes less than one gigabyte. Context is not the bottleneck for this model; only the weights are.
• Total size: Hugging Face counts 763 billion parameters. The difference to the backbone probably comes from additional components such as the Engram tables, the vision encoder and the layers for speculative decoding. The two Engram files together are a little over 200 GB and can stay on the SSD.
• Already natively 4-bit: DeepSeek ships the experts in FP4 and the rest in FP8. The usual savings from 4-bit quantization therefore do not apply to this model.

How good are the quants?

Getting a 552B model onto a single 256 GB machine requires going below 3 bits per weight. What matters is how much quality that costs. The following values come from the respective creators' publications and are measured against the native 1017 model. Top-1 means how often the quant picks the same next token as the original. KLD measures the divergence of the probability distributions; lower is better.

The measurement methods differ between creators, so the numbers are not comparable to the decimal. Three findings still hold:

1. Format matters more than bit count. EXL3 builds around 2.8 bpw are practically at original level. For the sudoingx build, HumanEval+, MBPP+ and IFEval were within the noise of the native model. Simple MLX quants of comparable size deviate by an order of magnitude more.
2. Non-English text suffers first. In the Rapid-MLX build, the deviation stays low for code (KLD 0.055) and math (0.063) but rises to 0.14 to 0.18 for general chat. Non-English text and niche knowledge drift the most. In the REAP build, agreement on multilingual text drops to 52%. If you work in other languages, test quants with your own prompts in that language.
3. The fastest Mac numbers come from a package without published quality metrics. Like EXL3, the Sushi-STQ format stores the experts with the efficient trellis method. That suggests good quality per bit. But 1.75 bpw is very little. Until KLD values are published, measure it yourself.

The engines: status at the end of October 2026

The pace is remarkable. DeepSeek released 1017 on October 17. Five days later it ran split across two Macs in mlx-serve, and on October 28 the plugin system arrived with the Sushi kernel. Within a few days, the plugin repository added several optimizations that each sped up long-prompt reading by 4 to 16%. Important for assessing the risk: the foundation, RDMA in macOS and jaccl in MLX, comes officially from Apple. The community projects build on it.

Measured: one M5 Ultra versus the cluster

A single M5 Ultra with 256 GB

The most robust numbers come from the plugin repository's pull requests. Conditions: M5 Ultra with 256 GB, macOS 27.2, Sushi-STQ 1.75 bpw package, all 384 experts per layer in RAM, one request at a time.

Around 198 GB are available for the experts, enough for all 384 experts in every layer. With a 3.0 bpw package in mlx-serve's native model code, earlier measurements showed 56.8 tok/s without and 48 to 82 tok/s with speculative decoding, depending on the type of text. The mlx-serve developers have since published a benchmark on X using the DeepSeek-V4.1-Flash-1017-iQ-MLX-3.0bpw package on an M5 Ultra with 256 GB. Conditions according to the screenshot: greedy decoding, one request, median of three runs after a warm-up.

The screenshot does not show the context length of the prefill measurement. For agent work and non-English text, the novel-text value is the more realistic one.

Independently, a community build based on TensorFold confirms the order of magnitude: on a single 256 GB M5 Ultra it measures 67.5 to 94.4 tok/s with speculative decoding and 1,093 to 1,392 tok/s prefill on a cold prompt, including image input, tool calls and 1M tokens of context.

What cluster measurements with DeepSeek V4.1 show so far

We found no published measurement for two M5 Ultras with working RDMA as of publication. There are, however, three informative data points:

• 2× M5 Ultra 256 GB, oMLX (bug report from October 20): Initially 1 tok/s, then 16.7 tok/s after fixes in version 0.7.2. This value is not representative, though: according to the maintainer, the connection ran over a normal TCP ring instead of RDMA, because the address was on the Thunderbolt Bridge instead of the Thunderbolt port. With 256 GB per Mac, only tensor parallelism fit for this package, and the Engram tables stay on the SSD.
• 2× M3 Ultra over Thunderbolt, oMLX 0.7.3rc1 (release notes, oQ4e package): Pipeline 32.4 to 34.3 tok/s, tensor parallelism 24.3 to 25.4 tok/s. According to the same release notes, a single M3 Ultra reaches 34.3 tok/s with the same package. Here the cluster is at best equally fast, and slower with tensor parallelism.
• 3× M5 Max with 128 GB each, native weights, RDMA (experimental setup): 35.6 tok/s for a single request.

According to its code description, mlx-serve splits DeepSeek V4.1 in a cluster by layers, i.e. via pipeline. According to Apple's measurements, that brings no speedup when generating.

The assessment: For DeepSeek V4.1 there is so far no public evidence that a cluster is faster than a single machine. A plausible reason: with only 16 billion active parameters per token, the compute per layer is small, while the synchronization overhead between machines stays the same. Apple sees the same effect in its measurements: dense models scale better across machines. New measurements appear almost daily. With cluster numbers, it is always worth checking three details: did the link run over RDMA, which package was used, and at what context length was it measured? For 1017, the conclusion stands: the model fits on one machine, and that is where it runs fastest today.

For comparison on the NVIDIA side: two DGX Sparks run 1017 with the EXL3 build at 2.8 bpw and near-original quality at around 104 tok/s for code, 58 for prose and 144 for structured output. We measured how a 4× DGX Spark cluster performs in self-hosting here (in German).

512 GB or two times 256 GB?

Apple opened orders for the 512 GB configuration on October 28, with a delivery time of 7 to 8 weeks. In the US it costs $22,299, exactly $11,500 more than the same configuration with 256 GB. In Germany, an M5 Ultra with 36-core CPU, 80-core GPU, 256 GB and 1 TB SSD costs €12,429 based on the upgrade prices listed by Macwelt. We could not verify a euro price for the 512 GB version as of publication.

A 512 GB machine therefore costs about as much as two 256 GB machines.

The comparison graphic "Two 256GB M5 Ultras beat one 512GB"

A graphic circulating on X claims that two 256 GB machines clearly beat one 512 GB machine: around 215 to 245 instead of 135 TFLOPS, 1.9 to 2.2 instead of 1.2 TB/s, 75 to 100 instead of 45 to 60 tok/s for DeepSeek V4.1 Flash, and 55 to 70 instead of 33 to 40 tok/s for GLM-5.3. We share the general direction: for models that really need 512 GB and whose engine handles tensor parallelism over RDMA, two 256 GB machines are the better choice. As a measurement, however, the graphic does not hold up:

• No measurement details: Engine, quantization and context length are missing. The values are given with a tilde and as ranges, and the cluster values are consistently about 1.7 times the single-machine values. That is exactly the factor Apple measures for dense models. So it is apparently a projection.
• The single-machine value for DeepSeek V4.1 does not match the measurements: An M5 Ultra with 256 GB, the same chip as in the 512 GB machine, reaches 92 tok/s with a 3-bit package. A 512 GB machine would be just as fast with the same package. The 45 to 60 tok/s only make sense if a higher-precision and therefore slower variant runs on the 512 GB machine. But then the graphic compares different quantizations.
• The cluster value matches a single 256 GB machine: A single M5 Ultra with 256 GB already measures the 75 to 100 tok/s claimed for two machines. For DeepSeek V4.1, the second machine therefore currently brings no speed, only room for higher precision.

A common misconception

A 512 GB machine does not make the same model faster. Chip and memory bandwidth are identical to the 256 GB model. If you use the extra memory for higher precision, generation even gets slower, because more bytes have to be read per token. 1017 at 4 bits therefore runs slower on the 512 GB machine than at just under 3 bits on the 256 GB machine.

What is left for the 512 GB machine

If you are willing to run 750B-class models at 3 to 3.5 bits, two 256 GB machines give you the same capacity. The only real advantage left for the 512 GB machine is independence from cluster support: it runs every model the engine supports at all. Add to that the upgrade path to 1 TB with two 512 GB machines, for example for Muse Spark Open at 4 bits.

Will the engines get better? Yes, but not everywhere

Software progress helps the two options in different ways.

What engines can improve:

• Better quants: The same quality with fewer bits makes 256 GB sufficient for ever larger models. That argues against paying extra for 512 GB.
• SSD streaming: Engines read parts of the model that are too large from the SSD on demand. This is how mlx-serve runs GLM-5.3 with 744 billion parameters on a single 256 GB Mac.
• Cluster support for more models: Every extension makes two 256 GB machines more useful.
• More efficient tensor parallelism: more gain per additional machine.

What engines cannot change:

• Memory bandwidth per Mac: Generation is limited by it, and 1.2 TB/s stays 1.2 TB/s.
• The Thunderbolt link: Latency and a practical throughput of 50 to 60 Gbit/s limit how well tensor parallelism scales.
• The number of machines: Common engines such as exo require powers of two for tensor parallelism, i.e. 2 or 4 machines. A full mesh is limited by the Thunderbolt ports.
• Model growth: Muse Spark Open has 816 billion parameters, DeepSeek V4 Pro had 1.6 trillion, and DeepSeek has signaled a V4.1 Pro.

On balance, software progress favors buying 256 GB machines step by step rather than the 512 GB machine.

SSD streaming and wear

Anyone running models via SSD streaming rightly asks whether it wears out the SSD. The short answer: streaming itself does not. Flash cells wear out from writes, not reads. Frequent reads can minimally affect neighboring cells. The controller then refreshes affected blocks in the background, which is negligible compared with normal use.

Writes happen in three places: when swapping memory, when saving conversation states in the prompt cache, and when downloading models. Apple does not publish endurance figures. We therefore assume a typical TLC SSD value of about 600 TB written per TB of capacity, so around 2,400 TB for a 4 TB SSD. This is an assumption, not a manufacturer specification.

The cache values are rough example calculations. For DeepSeek V4.1, 128,000 tokens of conversation state are only about 110 MB thanks to the small KV cache.

The real risk is swap caused by a memory budget set too tight. Since the SSD in the Mac Studio cannot simply be replaced by yourself, it pays to check:

1. After setup, run brew install smartmontools and smartctl -a disk0 and note "Percentage Used" and "Data Units Written".
2. Measure again after four weeks. If "Percentage Used" rises by more than 1% per month, swap is usually the cause.
3. Watch sysctl vm.swapusage during inference and lower the engine's memory budget if the value keeps growing.

The bigger downside of streaming is speed anyway. Generation speed then depends on the SSD's read throughput. With 1017, Sushi reaches 76.6 tok/s for code and 49.3 for prose on an M5 Max with 128 GB via streaming; an M5 Ultra with the whole model in RAM is clearly faster.

Our recommendation

1. Buy one M5 Ultra with 256 GB now (36-core CPU, 80-core GPU, at least 4 TB SSD for several model packages). It is fully sufficient for DeepSeek V4.1 Flash 1017, with mlx-serve and the sushi_ds41 plugin or with Sushi directly.
2. Check quant quality yourself, especially in languages other than English. Until KLD values are published for the fast STQ packages, compare against the DeepSeek API with your own prompts.
3. Buy the second 256 GB machine only when two conditions are met: a model you need no longer fits on one machine, and your engine demonstrably supports tensor parallelism over RDMA for it.
4. Choose the 512 GB machine only as insurance: if you want no dependence on cluster support at all, or if you plan two 512 GB machines for 1 TB from the start.
5. Set up operations cleanly: run headless, raise the GPU limit, disable the Thunderbolt Bridge, assign fixed IP addresses to the Thunderbolt port and monitor swap.

Frequently asked questions

Can two 256 GB Mac Studios be combined into one 512 GB machine? Not in the sense of shared memory for arbitrary programs. Inference engines such as MLX, mlx-serve or oMLX can, however, split a model across both machines, over Thunderbolt 5 with RDMA from macOS 26.2. Around 2 × 250 GB are usable in practice. Whether it works depends on whether the engine supports the specific model in a cluster.

Is a two-Mac cluster faster than a single Mac? Only with tensor parallelism over real RDMA. In that case Apple measures roughly a factor of 1.6 to 1.8 on the M5 Ultra. With pipeline parallelism, generation speed stays practically the same. For DeepSeek V4.1, the cluster measurements published so far show no advantage over a single machine. A single M5 Ultra runs the model at around 90 to 100 tok/s.

Does DeepSeek V4.1 Flash 1017 run on a single 256 GB Mac Studio? Yes. With a 3-bit package, the mlx-serve developers measure 92 tok/s when generating and almost 3,900 tok/s when reading on a 256 GB M5 Ultra. The Sushi-STQ package at 1.75 bpw reaches around 100 tok/s. The Engram tables stay on the SSD. No quality metrics have been published for the Sushi-STQ packages yet.

Is the extra cost for 512 GB worth it? For most people, no. The surcharge roughly equals a second 256 GB machine, and the 512 GB machine does not make the same model faster. It is worth it if you want to run 750B-class models at higher precision without any dependence on cluster support, or if you plan to expand to 1 TB with two 512 GB machines.

Does streaming kill the SSD? No. Streaming only reads, and reads practically do not wear flash memory. Constant swapping due to a too-tight memory budget is what matters. smartctl and sysctl vm.swapusage make this easy to monitor.

How much does quantization hurt quality? It mainly depends on the format. EXL3 builds around 2.8 bpw agree with the original DeepSeek V4.1 Flash 1017 about 96% of the time, simple MLX 2-bit quants only about 80%. Non-English languages lose quality first.

Sources

• Apple Machine Learning Research: Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio (Oct 15, 2026) – https://machinelearning.apple.com/research/mlx-distributed-inference
• computingforgeeks: Cluster Local LLMs – Mac Studio vs Strix Halo vs Two DGX Sparks – https://computingforgeeks.com/?p=178136
• mlx-serve release v26.10.3 (Oct 22, 2026) – https://github.com/ddalcu/mlx-serve/releases/tag/v26.10.3
• mlx-serve PR #895: DeepSeek-V4.1 across a Thunderbolt pair – https://github.com/ddalcu/mlx-serve/pull/895
• mlx-serve PR #911: DeepSeek-V4.1 decode on M5 Ultra – https://github.com/ddalcu/mlx-serve/pull/911
• mlx-serve PR #1002: Outside model plugins (Oct 28, 2026) – https://github.com/ddalcu/mlx-serve/pull/1002
• CommunityServe/plugins (sushi_ds41) – https://github.com/CommunityServe/plugins
• CommunityServe/plugins PR #10 and #11: prefill and decode measurements, M5 Ultra 256 GB – https://github.com/CommunityServe/plugins/pull/10 and https://github.com/CommunityServe/plugins/pull/11
• Sushi v1.4.0 – https://newreleases.io/project/github/beamivalice/sushi/release/v1.4.0
• oMLX issue #4720: clustering DeepSeek V4.1-1017 on 2× M5 Ultra 256 GB – https://github.com/jundot/omlx/issues/4720
• oMLX releases (0.7.3rc1, cluster measurements 2× M3 Ultra) – https://github.com/jundot/omlx/releases
• MLX discussion #4491: DeepSeek V4.1 Flash on three M5 Max Macs – https://github.com/ml-explore/mlx/discussions/4491
• BobClawblaw/deepseek-v41-flash-mlx-m5-ultra (TensorFold build for one M5 Ultra) – https://github.com/BobClawblaw/deepseek-v41-flash-mlx-m5-ultra
• sudoingX/dgx-spark-deepseek-v41 releases (EXL3 quality and DGX Spark speed) – https://github.com/sudoingX/dgx-spark-deepseek-v41/releases
• rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX (quality measurement) – https://huggingface.co/rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX
• patrickbdevaney/DeepSeek-V4.1-Flash-1017-REAP192-MXFP4 – https://huggingface.co/patrickbdevaney/DeepSeek-V4.1-Flash-1017-REAP192-MXFP4
• LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 (native FP4 experts) – https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4
• aiweekly: DeepSeek posts V4.1-Flash: 552B MoE, 8B active, 1M context – https://aiweekly.co/alerts/deepseek-posts-v41-flash-552b-moe-8b-active-1m-context
• TensorFold – https://github.com/ashhart/TensorFold and fork with DGX Spark branch https://github.com/Elevatormusic/TensorFold
• Macworld: The Mac Studio with 512GB of RAM is shipping – https://macworld.com/article/3271691/the-mac-studio-with-512gb-of-ram-is-shipping-and-it-costs-new-car-money.html
• Appleosophy: delivery time of the 512 GB Mac Studio – https://appleosophy.com/2026/10/28/mac-studio-with-512gb-of-ram-now-available-starts-at-22299/
• Macwelt: Mac Studio M5 Max and M5 Ultra, prices in Germany – https://www.macwelt.de/article/3220005/mac-studio-m5-max-m5-ultra.html
• iThinkDiff: 512GB Mac Studio and Muse Spark Open – https://www.ithinkdiff.com/512gb-mac-studio-m5-ultra-price-22299/
• mlx-serve developers: benchmark screenshot DeepSeek-V4.1-Flash-1017-iQ-MLX-3.0bpw on M5 Ultra 256 GB, published on X (October 2026)
• Comparison graphic "Two 256GB M5 Ultras beat one 512GB", circulated on X (October 2026, author unknown)