GPUFits
MoE Model Hardware Requirements: Total vs Active Parameters
updated 2026-09-10 · verified 2026-09-10
Every MoE (Mixture of Experts) model card carries two parameter numbers: total and active. The most common beginner mistake is grabbing the wrong one — sizing VRAM off active parameters and discovering the model doesn’t fit, or estimating speed off total parameters and assuming it will crawl. Each number governs exactly one thing: capacity follows total parameters, speed follows active parameters. This article runs that rule into concrete numbers for three representatives — Qwen3-30B-A3B, gpt-oss-120b, and DeepSeek-R1. All figures use the same formulas as our VRAM calculator: weights = parameters × bytes per weight, plus KV cache and ~1.5GB of runtime overhead. Every speed figure is a theoretical estimate; real results vary with framework and drivers, ±30%.
MoE in 60 seconds
A dense model passes every token through all of its parameters. A MoE model replaces the feed-forward layers with dozens or hundreds of “experts,” and a router picks a handful of them per token:
| Model | Total params | Active params | Expert structure |
|---|---|---|---|
| Qwen3-30B-A3B | 30.5B | 3.3B | 128 experts, 8 per token |
| gpt-oss-20b | 20.9B | 3.6B | 32 experts, 4 per token |
| gpt-oss-120b | 116.8B | 5.1B | 128 experts, 4 per token |
| DeepSeek-R1 | 671B | 37B | 256 routed + 1 shared, 8 per token |
The design decouples knowledge capacity from per-token compute: total parameters decide how much the model knows, active parameters decide how much bandwidth each token costs. The two hardware corollaries are the title of this article.
Rule 1: VRAM follows total parameters
Why can’t Qwen3-30B-A3B — which only activates 3.3B parameters — fit in an 8GB card? Because which 8 experts fire next is decided at generation time, per token. You can’t know who gets called, so all 128 experts must stay resident: all 30.5B parameters, no exceptions. “Loading only the active parameters” is not something current inference runtimes do.
Running the formula (total = weights + KV cache at 8K context + 1.5GB overhead):
| Model | Q8_0 | Q4_K_M | Q3_K_M | Q2_K |
|---|---|---|---|---|
| Qwen3-30B-A3B | 34.7GB | 21.0GB | 17.6GB | 14.4GB |
| gpt-oss-20b | 24.1GB | 14.7GB | 12.4GB | 10.2GB |
| gpt-oss-120b | 126.3GB | 73.6GB | 60.5GB | 48.4GB |
| DeepSeek-R1 | 715.9GB | 413.1GB | 337.6GB | 268.0GB |
| For comparison: Qwen3-32B (dense) | 38.5GB | 23.7GB | 20.0GB | 16.6GB |
Tier by tier, against usable VRAM (nominal for discrete GPUs, ×0.75 for Apple unified memory):
- gpt-oss-20b: 14.7GB at Q4 — exactly “tight” on a 16GB card (RTX 4070 Ti Super). OpenAI’s native MXFP4 build is smaller still; the official line is that it runs within 16GB. On 12GB you’re down to Q2 (10.2GB, tight) with obvious quality loss.
- Qwen3-30B-A3B: 21.0GB at Q4 fits a single 24GB card (3090/4090), tight. A 16GB card needs Q2 (14.4GB). A 32GB RTX 5090 runs Q5_K_M (24.1GB) comfortably, and a 48GB Mac mini M4 Pro (~36GB usable) swallows even Q8.
- gpt-oss-120b: the most-asked model in this family. 73.6GB at Q4_K_M — comfortable on 4×24GB (96GB), exactly tight on one A100/H100 80GB. A 96GB Mac Studio (~72GB usable) misses Q4 by 1.6GB and lands on Q3 (60.5GB, tight). OpenAI’s native MXFP4 build has roughly 60GB of weights, which is why they can claim “runs on a single 80GB GPU” — both sets of numbers are right; just don’t mix the two pipelines. Q2_K at 48.4GB is comfortable on the 96GB Mac but still a hair over dual 24GB cards.
- DeepSeek-R1: ~413GB at Q4 — 18× 3090 or 6× A100 80GB. That’s a different engineering league, covered in the dedicated guide.
Look at the last table row again: within the “30B class,” the MoE model saves only 2.7GB over the dense 32B at Q4. MoE doesn’t solve the capacity problem — it solves the speed problem.
Rule 2: speed follows active parameters
Generation is bandwidth-bound: every token produced requires reading the participating weights from memory once. The theoretical ceiling is roughly bandwidth × 0.75 ÷ weight bytes read per token. A dense model reads everything; a MoE model reads only the activated slice. At Q4:
| Model | Read per token | RTX 4090 | RTX 3060 | M3 Ultra 96GB |
|---|---|---|---|---|
| Qwen3-30B-A3B | 2.0GB | ~374 tok/s | ~134 tok/s | ~304 tok/s |
| gpt-oss-20b | 2.2GB | ~343 tok/s | ~122 tok/s | ~279 tok/s |
| gpt-oss-120b | 3.1GB | ~242 tok/s | ~86 tok/s | ~197 tok/s |
| DeepSeek-R1 | 22.7GB | ~33 tok/s¹ | ~12 tok/s¹ | ~27 tok/s¹ |
| For comparison: Qwen3-32B (dense) | 20.1GB | ~38 tok/s | ~13 tok/s | ~31 tok/s |
(Theoretical estimates, ±30%. ¹ R1’s row is pure math — none of these cards fit it; see below.)
Two comparisons worth memorizing:
- Same-class fight: Qwen3-30B-A3B and dense Qwen3-32B occupy nearly identical VRAM, but per-token reads differ 10× — and so does theoretical speed on the same card. That 10× is the entire point of MoE.
- Cross-class reversal: gpt-oss-120b is a 117B model that theoretically hits ~197 tok/s on a Mac Studio M3 Ultra — five times faster than a dense 32B on an RTX 4090. “Hundreds of billions of parameters” and “too slow to use” are not linked, provided you can pay the 74GB admission fee.
Why MoE is bandwidth-friendly
Read the formula backwards: generation speed = bandwidth ÷ bytes per token. There are only two ways to go faster — buy more bandwidth (expensive), or shrink the bytes per token (which is exactly what MoE does).
That matters most on three kinds of hardware:
- Apple unified memory. Apple Silicon offers a third to half the bandwidth of same-price GPUs, so dense models feel sluggish on Macs. MoE compresses per-token reads to 2–3GB, and suddenly 273–819GB/s is plenty. gpt-oss-120b is nearly the perfect M3 Ultra match: 96GB just fits Q3 or MXFP4, and 819GB/s theoretically delivers ~197 tok/s (±30%).
- CPU offload. Partial offload collapses effective bandwidth to system memory (~80GB/s in our model). A dense 32B manages a theoretical ~3 tok/s there — unusable. Offloaded gpt-oss-120b still theoretically reaches ~19 tok/s, and Qwen3-30B-A3B ~30 tok/s (±30%). A big-RAM box running a MoE model is the classic “120B on a budget” build; the trade-offs are in Is CPU offload worth it?.
- Multi-GPU. Same-model cards add bandwidth (×0.85 penalty), and MoE lets all of it serve just the active parameters. But note: every token gets routed across cards, so interconnect matters more than with dense tensor parallelism, and real speeds take an extra haircut beyond the theoretical one.
Three MoE-specific traps
Trap 1: sizing VRAM off active parameters. “30B-A3B only uses 3B, so 8GB is fine” — wrong; you’re storing 30.5B. This is the single most frequent MoE mistake. Before buying anything, run total parameters through the GPU checker.
Trap 2: assuming the KV cache is “activated” too. KV cache has nothing to do with experts — it’s set by layers, KV heads, and context length. The good news: all three MoE representatives have small KV footprints. gpt-oss-120b needs just 0.6GB at 8K by the standard formula (half its layers use sliding-window attention, so real usage is even lower), and ~9.7GB at 128K. Qwen3-30B-A3B at its full 32K context is ~3.2GB. R1’s MLA compresses 128K to ~9.2GB. With MoE, VRAM pressure lives almost entirely in the weights, and context planning is easy — details in our KV cache guide.
Trap 3: treating theoretical speed as a promise. The bandwidth formula is a ceiling, and ±30% is the dense-model disclaimer. MoE subtracts three more things: router compute, degraded memory locality from hopping between experts in VRAM, and framework maturity for MoE paths. gpt-oss-120b’s theoretical 242 tok/s on a 4090 doesn’t mean you’ll see 242 — use it for ranking, not acceptance testing.
Picking hardware, tier by tier
- 12GB (RTX 3060): gpt-oss-20b at Q2 (10.2GB, tight). MoE at this tier is underwhelming — a dense 8B at Q4 is the honest choice.
- 16GB (4070 Ti Super): gpt-oss-20b at Q4 (14.7GB, tight) — the first time a 16GB card runs a “20B-class” model fast. A genuinely new sweet spot created by MoE.
- 24GB (3090/4090): Qwen3-30B-A3B at Q4 (21.0GB, tight), theoretical 350+ tok/s — near the ceiling of the single-card experience. gpt-oss-120b is out of reach.
- 32GB (RTX 5090): Qwen3-30B-A3B at Q5 comfortably, or Q4 with the full 32K context.
- 48–96GB: gpt-oss-120b territory. 4×24GB, one A100 80GB, or a 96GB Mac Studio (Q3/MXFP4) are three very different budgets — run the cost calculator against API pricing before committing.
- Beyond that: for DeepSeek-R1, go straight to the dedicated guide. The conclusion hasn’t moved: distillates or API.
The one-line conclusion
MoE splits the hardware bill in two: you pay full price in VRAM, on total parameters, and you get a discount in speed, on active parameters. So MoE won’t help you fit a big model cheaply — it makes hardware that already fits a big model run 3–10× faster. A Mac Studio running gpt-oss-120b outpacing a GPU running a dense 32B is the rule made visible.
FAQ
How much VRAM does gpt-oss-120b need?
About 74GB total at Q4_K_M (71.5GB weights plus KV cache and overhead) — that means 4×24GB cards or one A100/H100 80GB, tight. OpenAI's native MXFP4 build has ~60GB of weights and officially runs on a single 80GB GPU. Q2_K lands near 48GB, which a 96GB Mac Studio can just handle.
Can a MoE model load only its active parameters into VRAM?
No. Which experts fire is decided per token at generation time, so every expert must stay resident. Active parameters determine speed, not capacity — gpt-oss-120b sizes like a 117B model but generates like a 5.1B one.
Qwen3-30B-A3B vs dense Qwen3-32B — what's the hardware difference?
VRAM is close (21GB vs 24GB at Q4), but bytes read per token differ 10× (2.0GB vs 20.1GB), so theoretical speed differs ~10× on the same card — roughly 374 vs 38 tok/s on an RTX 4090 (theoretical estimate, ±30%).
Are MoE models good candidates for CPU offload?
Much better than dense models. Offloaded, bandwidth collapses to system memory (~80GB/s): gpt-oss-120b still theoretically manages ~19 tok/s and Qwen3-30B-A3B ~30 tok/s, while a dense 32B drops to ~3 tok/s (theoretical estimate, ±30%).
Why do real MoE speeds fall short of the theory?
The bandwidth formula is a ceiling (±30%). MoE adds its own losses: router compute, worse memory locality from hopping between experts in VRAM, and per-token cross-GPU routing in multi-card setups. Expect real speeds to land below an equivalent dense model with the same active parameters.