GPUFits
How Much VRAM per Billion Parameters? (2026 Cheat Sheet)
updated 2026-08-04 · verified 2026-08-04
The single most useful number in local LLMs is VRAM per billion parameters. Learn this table once and you can size any model in your head in five seconds.
The cheat sheet (weights only, measured)
| Quantization | Bits per weight | GB per 1B params | vs FP16 |
|---|---|---|---|
| FP16 | 16.0 | 2.00 GB | — |
| Q8_0 | 8.5 | 1.06 GB | −47% |
| Q6_K | 6.6 | 0.82 GB | −59% |
| Q5_K_M | 5.7 | 0.71 GB | −64% |
| Q4_K_M | 4.9 | 0.61 GB | −69% |
| Q3_K_M | 4.0 | 0.50 GB | −75% |
| Q2_K | 3.2 | 0.40 GB | −80% |
These are not the theoretical llama.cpp bit-rates — they’re measured from real GGUF files (Llama-3.1-8B, verified 2026-08-04). Measured values run slightly above theory because embedding tables and metadata don’t compress as much. On larger models the gap shrinks.
The full formula
Total VRAM = weights (table above × params) + KV cache + overhead
- KV cache: roughly 0.5–4GB at 8K context for common models, grows linearly with context. Qwen3-32B goes from 2.1GB at 8K to 8.6GB at 32K.
- Overhead: ~1.5GB for the runtime itself (CUDA context etc.).
Worked examples (Q4_K_M, 8K context)
| Model | Weights | KV cache | Overhead | Total | Verdict |
|---|---|---|---|---|---|
| Llama 3.1 8B | 4.9 GB | 1.1 GB | 1.5 GB | 7.5 GB | comfy on 12GB |
| Qwen3 32B | 20.1 GB | 2.1 GB | 1.5 GB | 23.7 GB | tight on 24GB |
| Llama 3.3 70B | 43.2 GB | 2.7 GB | 1.5 GB | 47.4 GB | needs 2×24GB |
| gpt-oss-120b (MoE) | 71.5 GB | 1.2 GB | 1.5 GB | 74.2 GB | 96GB Mac or 4×24GB |
| DeepSeek-R1 671B | 410.7 GB | 0.6 GB¹ | 1.5 GB | 412.8 GB | cloud only |
¹ DeepSeek-R1 uses MLA attention, which compresses KV cache dramatically — see our KV cache guide.
Mental math shortcut
Rule of thumb: GB ≈ 0.6 × billions at Q4_K_M. 8B → ~5GB. 30B → ~18GB. 70B → ~43GB. Add ~2.5GB for KV cache and overhead at 8K context, more for longer contexts.
MoE models (DeepSeek-R1, gpt-oss, Qwen3-A3B) still need total parameters in memory — but they generate at the speed of their much smaller active parameter count. That’s why gpt-oss-120b feels fast despite needing 74GB.
FAQ
How much VRAM does a 7B or 8B model need?
About 4.9GB for weights at Q4_K_M, plus roughly 1.1GB of KV cache at 8K context and 1.5GB of runtime overhead — call it 7.5GB total. Any 12GB card handles it comfortably.
How much VRAM does a 70B model need?
About 43GB at Q4_K_M including KV cache and overhead — no single consumer GPU fits it. Two 24GB cards (48GB) run it tight; 64GB+ runs it comfortably.
Why does the cheat sheet use Q4_K_M as the default?
Q4_K_M is the community standard: it cuts VRAM by ~70% vs FP16 with a quality loss most users can't detect in chat. Every major runtime (llama.cpp, Ollama, LM Studio) ships it as the default download.