GPUFits
Models / Qwen3 8B
Qwen3 8B VRAM Requirements & GPU Pairing
Qwen3 8B is the quality benchmark of the 8B tier: 8.2B dense parameters, 36 layers, 8 KV heads, and a genuine Apache-2.0 license — the most practical difference from the Llama family. It supports hybrid thinking/non-thinking modes; in thinking mode its math and code abilities lead the class, and Chinese comprehension is home turf. The 36-layer design puts the 8K KV cache at ~1.2GB, marginally above Llama 3.1 8B's 1.1GB.
The trade-off is context length: native 32K (40,960 in config, YaRN-extensible to 131K), which is less comfortable for long-document RAG than Llama 3.1's 128K. VRAM needs are nearly identical to Llama 3.1 8B — Q4_K_M @8K totals ~7.7GB, comfortable on 12GB, with Q8_0 (~11.4GB) fitting 16GB cards. Bottom line: if one 8B model must serve Chinese or reasoning-heavy work, pick this; for very long context or the richest English tutorial ecosystem, pick Llama 3.1 8B.
Architecture Specs
| Total parameters | 8.2B |
| Active parameters | 8.2B |
| Layers | 36 |
| KV heads | 8 |
| Head dim | 128 |
| Native context | 32K |
| License | apache-2.0 |
| KV bytes per token (fp16) | 144.0 KB |
VRAM by Quant Tier (@8K context, fp16 KV)
| Quant | Weights | Total (incl. 1.5GB runtime overhead) |
|---|---|---|
| Q8_0 | 8.7 GB | 11.4 GB |
| Q6_K | 6.7 GB | 9.4 GB |
| Q4_K_M | 5.0 GB | 7.7 GB |
| Q3_K_M | 4.1 GB | 6.8 GB |
| Q2_K | 3.2 GB | 6.0 GB |
KV cache and runtime overhead do not change across quants; longer contexts grow the KV part linearly — adjust context length in the GPU checker tool.
GPU Verdict Matrix (Q4_K_M @8K)
| GPU | Usable VRAM | Verdict | Recommended quant | Theoretical speed | |
|---|---|---|---|---|---|
| RTX 3060 12GB | 12.0 GB | Comfortable | Q8_0 | ≈31 tok/s | Try it → |
| RTX 3090 | 24.0 GB | Comfortable | FP16 | ≈43 tok/s | Try it → |
| RTX 4070 Ti Super | 16.0 GB | Comfortable | Q8_0 | ≈58 tok/s | Try it → |
| RTX 4090 | 24.0 GB | Comfortable | FP16 | ≈46 tok/s | Try it → |
| RTX 5090 | 32.0 GB | Comfortable | FP16 | ≈82 tok/s | Try it → |
| RTX A6000 | 48.0 GB | Comfortable | FP16 | ≈35 tok/s | Try it → |
| A100 80GB | 80.0 GB | Comfortable | FP16 | ≈93 tok/s | Try it → |
| H100 80GB | 80.0 GB | Comfortable | FP16 | ≈153 tok/s | Try it → |
| RX 7900 XTX | 24.0 GB | Comfortable | FP16 | ≈44 tok/s | Try it → |
| Mac mini M4 Pro (48GB) | 36.0 GB | Comfortable | FP16 | ≈12 tok/s | Try it → |
| Mac Studio M4 Max (64GB) | 48.0 GB | Comfortable | FP16 | ≈25 tok/s | Try it → |
| Mac Studio M3 Ultra (96GB) | 72.0 GB | Comfortable | FP16 | ≈37 tok/s | Try it → |
Theoretical speed = bandwidth × 0.75 ÷ per-token weight bytes (active params for MoE); real-world results vary with framework/driver/CPU, ±30%.
FAQ
- How much VRAM does Qwen3 8B need?
- Q4_K_M @8K totals ~7.7GB — comfortable on 12GB. Q8_0 is ~11.4GB (16GB card); FP16 is ~19.1GB (24GB card).
- Is Qwen3 8B's 32K context enough?
- Fine for chat, code, and ~10k-word documents. For book-length PDFs or large codebases, use a native-128K model (Llama 3.1 8B, Mistral Small 24B) or enable YaRN extension to 131K with some quality loss.
- Can I use Qwen3 8B commercially?
- Yes. Apache-2.0 permits free commercial use, modification, and redistribution — no MAU clauses like the Llama community license.
Related guides
- Quantization Explained: Q4 vs Q8 and What You Actually Lose
- Best GPU for Local LLMs in 2026: Every Budget Tier
- KV Cache Explained: Why Long Contexts Eat Your VRAM
Try Qwen3 8B in the GPU compatibility checker →
Data verified 2026-09-01