Self-Hosted AI vs Cloud AI (2026): What 64 Benchmarked Coding Tasks Actually Cost

Our Recommendation

The honest answer is that self-hosting is a sovereignty purchase, not a savings purchase — and the benchmark says so plainly. The authors' own conclusion on whether to buy GPUs is 'probably not to save money'. Start with the number that decides it: utilisation. You size hardware for your peak and pay for it around the clock, while published enterprise figures for internal developer tooling put average GPU utilisation at 15-22%, rarely above 25-35%. Against that backdrop the thresholds are brutal and non-obvious. A 4xH200 box has to stay 89% busy for five years before owning it beats DeepSeek-V4-Flash's $2.13 API invoice for the same 64 tasks. An 8xB200 rack running GLM-5.2 needs only 15% to beat the frontier APIs. Same question, same workload, two answers an order of magnitude apart — because the API you are escaping is priced very differently depending on which model you were using. Now the part most cost comparisons omit: quality. On those 64 tasks the single-H200 setup running Qwen3.6 cost $0.43 in owned hardware against $57.67 in API tokens, which looks like a rout until you see it resolved 35.4% of tasks while the Opus 4.8 API resolved 62.5%. Buying the cheap box does not buy you the same work. At the top end the picture inverts: Kimi K3 resolved 86.4%, 24 points above both GLM-5.2 and Opus 4.8, but its 1.4TB of weights need an 8xB300 node, it served only 16 concurrent sessions, and its median task took 38 minutes — roughly eight times slower than the Claude Code baseline. The authors also caution that their SWE-bench Pro tasks may sit in K3's training data. Then concurrency, which is where developer experience quietly breaks. A DGX Spark handles one user and is slow even then. A single H200 comfortably serves 32 sessions. The 8xB200 rack serving a near-frontier model is back down to about eight developers before tasks start crawling. Cloud APIs simply do not have this ceiling. Our take: if your reason is data that cannot leave the building, a stack nobody can rate-limit, or a token bill that has become genuinely unpredictable, self-host — and size it against your measured peak, not your headcount. If your reason is saving money, measure your utilisation first, and consider renting before buying: renting was 35x cheaper than the API for Qwen3.6, 23% cheaper for GLM-5.2, and more expensive than DeepSeek's own API for DeepSeek. Cloud remains the right default for most teams, and the switch is worth revisiting every few months as agent adoption pushes your utilisation up.