Active Parameters (MoE)

In Mixture-of-Experts (MoE) models, active parameters are the model parameters that are actually computed when processing a single token. An MoE model consists of several specialized expert blocks; per token, a router activates only a small fraction of them. The meaningful metric is therefore the pair of total size and active size — roughly 125B total / 6B active for Qwen 3.8 Flash Next, 770B / 49B for Tencent Hy-4 Preview, about 18B active for GLM 5.3 Flash, and 2B for the compact Mini CPM5. The difference from a classic dense model: there, the number of parameters computed per token equals the total number. In MoE models, per-token compute scales with the active count, while memory demand is set by the total count — the two values describe different bottlenecks. Why the active count drives practice: it directly controls token throughput (more active parameters = more operations per token) and inference cost per million tokens, hence the API price, while the total count defines VRAM requirements. A model with 125B total and 6B active parameters computes per token like a small 6B model but offers the knowledge breadth of a much larger one — the efficiency frontier of today's open-weight line. For reading model sheets: the active count is not marketing gloss but the second, equally important core metric. Specs like 125B/6B or 770B/49B show that knowledge breadth and speed became achievable by decoupling total and active size — honest parameter pairs instead of single trillion-digit numbers.