GPUFits
Models / Qwen3 32B
Qwen3 32B VRAM Requirements & GPU Pairing
Qwen3 32B is the dense camp's backbone: all 32.8B parameters fire on every forward pass, 64 layers, 8 KV heads, Apache-2.0, native 32K context. It is the quality ceiling of a single 24GB card — Q4_K_M @8K totals ~23.7GB, a tight fit on RTX 3090/4090, and headroom requires dropping to Q3_K_M (~20GB). Only the 32GB RTX 5090 runs it comfortably, with room for Q6_K.
Speed is the dense tax: every token reads the full ~20GB of weights, so a 4090 theoretically manages ~38 tok/s — one tenth of its MoE sibling 30B-A3B. The 64 layers make the 8K KV ~2.1GB; budget VRAM for long contexts and consider q8 KV. The choice is simple: 30B-A3B for speed, 32B for dense knowledge consistency and predictable behavior. It is also the representative tier of the DeepSeek-R1 distillation ecosystem (R1-Distill-Qwen-32B and same-size peers), so reasoning fine-tunes are mature.
Architecture Specs
| Total parameters | 32.8B |
| Active parameters | 32.8B |
| Layers | 64 |
| KV heads | 8 |
| Head dim | 128 |
| Native context | 32K |
| License | apache-2.0 |
| KV bytes per token (fp16) | 256.0 KB |
VRAM by Quant Tier (@8K context, fp16 KV)
| Quant | Weights | Total (incl. 1.5GB runtime overhead) |
|---|---|---|
| Q8_0 | 34.9 GB | 38.5 GB |
| Q6_K | 26.9 GB | 30.6 GB |
| Q4_K_M | 20.1 GB | 23.7 GB |
| Q3_K_M | 16.4 GB | 20.0 GB |
| Q2_K | 13.0 GB | 16.6 GB |
KV cache and runtime overhead do not change across quants; longer contexts grow the KV part linearly — adjust context length in the GPU checker tool.
GPU Verdict Matrix (Q4_K_M @8K)
| GPU | Usable VRAM | Verdict | Recommended quant | Theoretical speed | |
|---|---|---|---|---|---|
| RTX 3060 12GB | 12.0 GB | Not feasible | needs 2 cards | — | Try it → |
| RTX 3090 | 24.0 GB | Tight fit | Q4_K_M | ≈35 tok/s | Try it → |
| RTX 4070 Ti Super | 16.0 GB | Not feasible | needs 2 cards | — | Try it → |
| RTX 4090 | 24.0 GB | Tight fit | Q4_K_M | ≈38 tok/s | Try it → |
| RTX 5090 | 32.0 GB | Comfortable | Q6_K | ≈50 tok/s | Try it → |
| RTX A6000 | 48.0 GB | Comfortable | Q8_0 | ≈17 tok/s | Try it → |
| A100 80GB | 80.0 GB | Comfortable | FP16 | ≈23 tok/s | Try it → |
| H100 80GB | 80.0 GB | Comfortable | FP16 | ≈38 tok/s | Try it → |
| RX 7900 XTX | 24.0 GB | Tight fit | Q4_K_M | ≈36 tok/s | Try it → |
| Mac mini M4 Pro (48GB) | 36.0 GB | Comfortable | Q6_K | ≈8 tok/s | Try it → |
| Mac Studio M4 Max (64GB) | 48.0 GB | Comfortable | Q8_0 | ≈12 tok/s | Try it → |
| Mac Studio M3 Ultra (96GB) | 72.0 GB | Comfortable | FP16 | ≈9 tok/s | Try it → |
Theoretical speed = bandwidth × 0.75 ÷ per-token weight bytes (active params for MoE); real-world results vary with framework/driver/CPU, ±30%.
FAQ
- Can a 24GB GPU run Qwen3 32B?
- Tight fit: Q4_K_M @8K is ~23.7GB, nearly filling 24GB. Keep context moderate and quantize KV to q8; for comfort get a 32GB RTX 5090, or drop to Q3_K_M (~20GB).
- Qwen3 32B vs 30B-A3B — what's the real difference?
- 32B is dense: all 32.8B params compute every token, ~38 tok/s theoretical on a 4090. 30B-A3B is MoE: 3.3B active, ~374 tok/s. Quality trades blows, speed differs 10× — pick “solid” or “fast”.
- What is Qwen3 32B's context length?
- Native 32K (40,960 in config), YaRN-extensible to 131K with some quality loss in the extended range. 64 layers × 8 KV heads make the 8K KV ~2.1GB — reserve VRAM for long contexts.
Related guides
- Quantization Explained: Q4 vs Q8 and What You Actually Lose
- MoE Model Hardware Requirements: Total vs Active Parameters
- Best GPU for Local LLMs in 2026: Every Budget Tier
Try Qwen3 32B in the GPU compatibility checker →
Data verified 2026-09-01