Updated: September 30, 2026 · by Michael Kerkhoff

Reviewed by Michael Kerkhoff, as of September 30, 2026

Best Open-Weight LLMs 2026: Self-Hosting Guide

Compare the best open-weight LLMs for self-hosting in 2026: Qwen3.5, GLM-5.2, gpt-oss-120b, DeepSeek-V4, Kimi K2.7 Code, Gemma 4, Mistral Small 4 and MiniMax M3 (as of July 2026) — parameter counts, context lengths, licenses and hardware requirements.

Top Picks

(01)

Qwen3.5 (Alibaba)

Best all-round open-weight family and the safest default: one architecture published across 0.8B, 2B, 4B, 9B, 27B, 35B-A3B and 122B-A10B, all under Apache 2.0. The flagship Qwen3.5-122B-A10B activates 10B of 122B parameters per token and ships a native 262,144-token context extensible to roughly 1M. The small variants are among the most widely used open-weight checkpoints on Hugging Face, which means a broad ecosystem of quantizations, fine-tunes and serving recipes. Start here unless a specific workload pushes you elsewhere.

General purpose across the whole size ladder — one family from laptop to multi-GPU nodeFree weights (Apache 2.0); you pay only for the GPUs you runAI-Native

(02)

GLM-5.2 (Z.ai / Zhipu)

Best open-weight model for long-horizon agentic and coding work. GLM-5.2 is the flagship sparse-attention MoE from Z.ai, released under a plain MIT license, and it is the first in the GLM line to sustain a solid 1M-token context rather than a nominal one. Its IndexShare attention reuses the same indexer across every four sparse layers, which, according to Z.ai's model card, cuts per-token FLOPs substantially at 1M context — that efficiency is the reason it is practical to self-host at all. Independent rankings such as Onyx's self-hosted LLM leaderboard place it among the strongest self-hostable coding models.

Long-horizon agents and coding — the strongest self-hostable coder in 2026Free weights (MIT); hosted API available via Z.ai if you do not want the clusterAI-Native

(03)

gpt-oss-120b / gpt-oss-20b (OpenAI)

Best capability-per-GPU and the easiest open-weight model to actually get into production. gpt-oss-120b has 117B parameters with 5.1B active and, thanks to MXFP4 post-training quantization of the MoE weights, runs on a single 80GB GPU (H100 or MI300X); gpt-oss-20b has 21B parameters with 3.6B active and fits in 16GB, which puts it on a workstation or a high-end laptop. Apache 2.0, fine-tunable — the 120b on a single H100 node, the 20b on consumer hardware. If your constraint is "one GPU we already own," this is the shortlist of one.

Best fit for constrained hardware — frontier-adjacent reasoning on a single GPUFree weights (Apache 2.0), no copyleft or patent strings attachedAI-Native

(04)

DeepSeek-V4 (Pro / Flash)

Best open-weight frontier reasoning if you have the cluster. DeepSeek-V4-Pro is a 1.6T-parameter MoE with 49B active, DeepSeek-V4-Flash is 284B with 13B active, and both support a full one-million-token context under an MIT license. The hybrid attention design (Compressed Sparse Attention plus Heavily Compressed Attention) is what makes the long context affordable: according to DeepSeek's model card, at 1M tokens V4-Pro needs only a fraction of the per-token inference FLOPs and KV cache of its predecessor V3.2. Pro is a rental-or-cluster proposition; Flash is the variant most teams will actually self-host.

Frontier reasoning and million-token document workFree weights (MIT); serious GPU budget or a hosted provider required for ProAI-Native

(05)

Kimi K2.7 Code (Moonshot AI)

Best open-weight coding agent when you want a model tuned for the agent loop rather than for chat. Kimi K2.7 Code is a 1T-parameter MoE with 32B active, 384 experts (8 selected plus 1 shared per token) and a 256K context, built on K2.6 and evaluated primarily inside agent harnesses rather than on static benchmarks. The catch to plan for: it ships under a Modified MIT license, not plain MIT or Apache — read the terms before you build a commercial product on it. Note also that the widely reported "Kimi K3 weights" had still not appeared in Moonshot's official Hugging Face org as of this guide's publication.

Agentic coding — tuned for long multi-step tool-use sessionsFree weights, but under a Modified MIT license — review before commercial use

(06)

Gemma 4 (Google DeepMind)

Best open-weight family for on-device and edge deployment. Gemma 4 ships five sizes — E2B, E4B, 12B, 26B-A4B and 31B — in both dense and MoE flavours, with a 256K context, multimodal text-and-image input (audio on E2B, E4B and 12B) and support for 140+ languages, all under Apache 2.0. If your deployment target is a phone, a laptop or an on-prem appliance rather than a GPU cluster, this family is designed for exactly that range.

On-device and edge — phones and laptops through to single-server deploymentsFree weights (Apache 2.0)AI-Native

(07)

Mistral Small 4 / Mistral Large 3

Best choice when EU data residency and sovereignty are hard requirements rather than nice-to-haves. Mistral Small 4 is a 119B-parameter model with 6.5B activated per token, a 256K context and a per-request reasoning_effort parameter, published under Apache 2.0; Mistral Large 3 (675B) is the Apache-2.0 flagship above it. A European vendor with permissive licensing is a materially different procurement conversation for regulated DACH industries — and Mistral's push into air-gapped and sovereign deployments makes it the pragmatic default there, even where a Chinese-lab model scores higher on a leaderboard.

EU-sovereign and regulated deployments — air-gapped, on-prem, data-residency-boundFree weights (Apache 2.0); commercial support and La Plateforme availableAI-Native

(08)

MiniMax-M3

Best open-weight model for high-throughput million-token workloads. MiniMax-M3 is a natively multimodal MoE with roughly 428B parameters and about 23B activated, built around MiniMax Sparse Attention (MSA) — which, according to MiniMax's model card, delivers much faster prefill and decode than its predecessor at 1M context. If your bottleneck is serving very long contexts cheaply rather than topping a reasoning leaderboard, that throughput profile is the differentiator. Licensed under MiniMax's own terms, so treat the license as a review item, not a formality.

High-throughput long-context serving and multimodal inputFree weights under MiniMax model licence — review terms before commercial use

Comparison Table

Name	Best For	Architecture & Context	Hardware Fit	License & Cost	Fully Permissive License
(01) Qwen3.5 (Alibaba)	General purpose across the whole size ladder — one family from laptop to multi-GPU node	MoE + dense variants, 262K context (extensible ~1M), multimodal, vLLM/SGLang/Transformers	Laptop (0.8B–9B) to 8× GPU node (122B-A10B)	Free weights (Apache 2.0); you pay only for the GPUs you run	✓ Yes
(02) GLM-5.2 (Z.ai / Zhipu)	Long-horizon agents and coding — the strongest self-hostable coder in 2026	Sparse-attention MoE with IndexShare, 1M-token context, MTP speculative decoding, FP8 weights available	Multi-GPU node (FP8 build recommended)	Free weights (MIT); hosted API available via Z.ai if you do not want the cluster	✓ Yes
(03) gpt-oss-120b / gpt-oss-20b (OpenAI)	Best fit for constrained hardware — frontier-adjacent reasoning on a single GPU	MoE with native MXFP4 quantization, harmony response format, vLLM/Ollama/llama.cpp support	Single 80GB GPU (120b) or 16GB workstation/laptop (20b)	Free weights (Apache 2.0), no copyleft or patent strings attached	✓ Yes
(04) DeepSeek-V4 (Pro / Flash)	Frontier reasoning and million-token document work	MoE with hybrid CSA/HCA attention, 1M context, DSpark speculative decoding, 32T pre-training tokens	Large cluster (Pro, 1.6T) or mid-size multi-GPU node (Flash, 284B)	Free weights (MIT); serious GPU budget or a hosted provider required for Pro	✓ Yes
(05) Kimi K2.7 Code (Moonshot AI)	Agentic coding — tuned for long multi-step tool-use sessions	MoE, 1T total / 32B active, 384 experts, 256K context, SwiGLU	Large multi-GPU cluster	Free weights, but under a Modified MIT license — review before commercial use	– No
(06) Gemma 4 (Google DeepMind)	On-device and edge — phones and laptops through to single-server deployments	Dense + MoE variants, 256K context, multimodal (text/image, audio on small sizes), 140+ languages	Phone/laptop (E2B–12B) to single server (26B-A4B, 31B)	Free weights (Apache 2.0)	✓ Yes
(07) Mistral Small 4 / Mistral Large 3	EU-sovereign and regulated deployments — air-gapped, on-prem, data-residency-bound	119B / 6.5B active (Small 4), 256K context, per-request reasoning_effort, vLLM-ready	Multi-GPU node (Small 4) to cluster (Large 3, 675B)	Free weights (Apache 2.0); commercial support and La Plateforme available	✓ Yes
(08) MiniMax-M3	High-throughput long-context serving and multimodal input	Sparse-attention MoE (MSA), ~428B total / ~23B active, 1M context, natively multimodal	Multi-GPU node to cluster	Free weights under MiniMax model licence — review terms before commercial use	– No

← Scroll horizontally to see all columns

How to Choose

(01)
Start from your hardware, not from the leaderboard. The single most useful filter in 2026 is "what fits on the GPUs we already have." One 80GB card means gpt-oss-120b. A 16GB workstation means gpt-oss-20b or a Qwen3.5/Gemma 4 small variant. A multi-GPU node opens GLM-5.2, Qwen3.5-122B-A10B, Mistral Small 4 and DeepSeek-V4-Flash. A 1.6T model like DeepSeek-V4-Pro is a cluster or a rented endpoint, not a self-hosting decision.
(02)
Read the license before you read the benchmarks. "Open weights" is not one thing. Apache 2.0 (Qwen3.5, gpt-oss, Gemma 4, Mistral) and MIT (GLM-5.2, DeepSeek-V4) are genuinely permissive. Modified MIT (Kimi K2.7 Code) and bespoke vendor licences (MiniMax-M3) carry conditions that matter once you ship a commercial product. Have legal look at the LICENSE file in the repository, not at the badge on the model card.
(03)
Treat advertised context length as a capacity claim, not a quality claim. Almost every flagship now says 1M tokens. What differs is whether the attention architecture makes that context affordable to serve — GLM-5.2's IndexShare, DeepSeek-V4's hybrid CSA/HCA and MiniMax's MSA all exist specifically to cut the FLOPs and KV-cache cost of long context. Benchmark long-context quality on your own documents before you size the cluster.
(04)
Match the model to the loop, not to the average. A coding agent that runs 200 tool calls per task rewards a model post-trained inside an agent harness (Kimi K2.7 Code, GLM-5.2). A RAG endpoint answering short questions at high volume rewards throughput and a small active-parameter count (gpt-oss, Qwen3.5-35B-A3B, MiniMax-M3). These are different purchases.
(05)
Standardise on one serving stack early. vLLM and SGLang cover the multi-GPU server case with PagedAttention, continuous batching and speculative decoding; Ollama and llama.cpp cover laptops and single-GPU boxes. Picking the stack first and the model second means you can swap weights later without rewriting your deployment — which is the entire point of going open-weight.
(06)
Do not swap one lock-in for another. Standing your whole stack on a single lab's weights — Meta, Alibaba or otherwise — recreates the dependency you left the API providers to escape. Put a gateway or an OpenAI-compatible proxy in front of your models so switching families is a config change, and keep at least one fallback family qualified on your own evals.
(07)
Verify releases against the primary source. In 2026 several outlets reported model weights as published on their announced date when the vendor's own Hugging Face organisation still had no such repository. Before you plan a migration around a release, check the org's model list on Hugging Face directly — third-party derivatives with matching names will fool a keyword search.

Frequently Asked Questions

(01)What is the difference between open-source and open-weight LLMs?

Open-weight means the trained model weights are downloadable under a licence that permits use — you can run, fine-tune and serve the model yourself. Open-source in the strict sense would additionally require the training data, training code and tokenizer to be published, which almost no frontier-scale model does. Qwen3.5, GLM-5.2, gpt-oss, DeepSeek-V4, Gemma 4 and Mistral are all open-weight; genuinely open-source models like OLMo and Pythia exist mainly in research contexts. Using the terms interchangeably is common but obscures the practical question, which is what the licence actually permits.

(02)What is the best open-weight LLM in 2026?

There is no single winner, but there is a reliable default: Qwen3.5, because Alibaba publishes the same architecture from 0.8B up to 122B-A10B under Apache 2.0, so one family covers a laptop prototype and a production cluster. For long-horizon agentic and coding work, GLM-5.2 (MIT, 1M context) is the strongest self-hostable option. For the best capability on a single GPU, gpt-oss-120b. For frontier reasoning at a million tokens, DeepSeek-V4. Pick by constraint, not by leaderboard position.

(03)What hardware do I need to self-host an open-weight LLM?

It ranges by three orders of magnitude. gpt-oss-20b (21B parameters, 3.6B active) runs within 16GB of memory, so a workstation or high-end laptop is enough. gpt-oss-120b (117B, 5.1B active) fits on a single 80GB GPU such as an H100 or MI300X thanks to MXFP4 quantization. Mid-size MoE models like Qwen3.5-122B-A10B or Mistral Small 4 typically want an 8-GPU node. Giant MoE models such as DeepSeek-V4-Pro (1.6T) or Kimi K2.7 Code (1T) need a real cluster, which is why most teams rent those through a provider instead of hosting them.

(04)Are open-weight models good enough to replace commercial APIs?

For a large share of production workloads, yes — and the honest framing is that this is now a cost, latency and control decision rather than a capability gap. Open-weight models handle classification, extraction, summarisation, RAG and much routine code generation at quality that is indistinguishable from a commercial API to the end user. The remaining gap shows up on the hardest reasoning and long-horizon agentic tasks. The pragmatic architecture is hybrid: self-host the high-volume work, route the hard tail to a frontier API, and put a gateway between your application and both.

(05)Can I use open-weight models commercially?

It depends entirely on the licence, and the licences genuinely differ. Apache 2.0 (Qwen3.5, gpt-oss-120b/20b, Gemma 4, Mistral Small 4 and Large 3) and MIT (GLM-5.2, DeepSeek-V4) allow commercial use without copyleft obligations. Kimi K2.7 Code ships under a Modified MIT licence and MiniMax-M3 under the vendor's own terms — both are usable commercially in the typical case but carry conditions you need to have read. Always open the LICENSE file in the actual repository before committing a product to a model.

(06)Why does self-hosting matter for GDPR and EU data residency?

Because self-hosting removes the transfer entirely. When the weights run on infrastructure you control — in an EU region, in your own data centre or fully air-gapped — no prompt or document ever leaves your legal boundary, which turns a complex international transfer assessment into an ordinary internal processing question. That is why regulated DACH organisations increasingly evaluate open-weight models first, and why Mistral's European origin plus Apache 2.0 licensing is a materially easier procurement conversation than a US or Chinese hosted API, independent of benchmark scores.

(07)Which serving stack should I use for self-hosted models?

For multi-GPU servers, vLLM and SGLang are the two defaults — both provide continuous batching, PagedAttention-style KV-cache management, tensor parallelism and speculative decoding, and both are what vendor model cards ship launch commands for. For laptops, single GPUs and edge boxes, Ollama and llama.cpp are the practical choice, with GGUF quantizations available for essentially every model in this guide. Choose the stack before the model: the whole benefit of open weights is that swapping families should be a configuration change.

(08)How do I keep an open-weight deployment current without re-architecting every quarter?

Decouple the three layers. Keep an OpenAI-compatible endpoint (vLLM, SGLang or a gateway such as LiteLLM) as the stable interface your application talks to, keep your evaluation suite in version control so a new checkpoint can be qualified in hours rather than weeks, and keep at least two model families qualified at any time. Release cadence in 2026 is roughly quarterly per lab, so the goal is not to chase every release — it is to make the swap cheap enough that you can take the ones that matter.

(09)What about the new Flash models (GLM-5.3-Flash, DeepSeek V4.1 Flash, Qwen3.8-Flash-Next)?

They postdate this guide's July snapshot and are covered in depth in our companion article Current Open-Weight LLMs & the Hardware They Run On (September 2026). Short version: GLM-5.3-Flash (320B MoE, ~18B active, MIT) is the strongest open-weight agentic coder of the Flash class; DeepSeek V4.1 Flash (552B MoE, MIT) is the price-performance pick; Qwen3.8-Flash-Next (125B+51B n-gram, ~6B active) previews the Qwen4 architecture. Native FP8 weights of the first two need an 8-GPU-class node; community EXL3 quants run them on 1-4 DGX Sparks with documented fidelity trade-offs.

(10)Is this guide still current after the August/September 2026 releases?

Yes for the established families it ranks (Qwen3.5, GLM-5.2, gpt-oss, DeepSeek-V4, Gemma 4, Kimi K2.7, Mistral, MiniMax-M3) — none of them was displaced by the Flash releases, which form a new class below the flagships rather than replacing them. The companion September 2026 article maps the new models onto concrete hardware tiers, from 16 GB laptops to 8-GPU nodes, and this guide's TLDR links to it. We re-verify the ranking quarterly.

Sources & Further Reading

Ready to start your AI project?

Book a free 30-minute consultation to discuss your requirements and find the right approach.

Book Consultation

Top Picks

Comparison Table

How to Choose

Frequently Asked Questions

Related Resources

Related Guides

AI Glossary

Our Services

Sources & Further Reading

Ready to start your AI project?