GPUFits
Models / gpt-oss-20b
gpt-oss-20b VRAM Requirements & GPU Pairing
gpt-oss-20b is OpenAI's first open-weight release in years: a 20.9B-total MoE (32 experts, 4 active per token) with just 3.6B active parameters, Apache-2.0 licensed. Half its layers use 2048-token sliding-window attention and headDim is 64, so the KV cache is tiny (~0.4GB at 8K); it natively supports MXFP4 low precision, the GGUF ecosystem is mature, and context reaches 128K.
Q4_K_M @8K totals ~14.7GB — a tight fit on 16GB cards, which is its killer feature: a single RTX 4070 Ti Super runs OpenAI-style reasoning (harmony output format, adjustable reasoning effort). Speed follows the 3.6B active params — a theoretical ~343 tok/s on a 4090. Note it reasons by default: dial effort down for simple tasks or you burn tokens. Ideal for users who want strong reasoning on a 16GB card; 24GB cards can step up to Q6_K (~19GB).
Architecture Specs
| Total parameters | 20.9B |
| Active parameters(MoE: VRAM follows total params, speed follows active params) | 3.6B |
| Layers | 24 |
| KV heads | 8 |
| Head dim | 64 |
| Native context | 128K |
| License | apache-2.0 |
| KV bytes per token (fp16) | 48.0 KB |
Sliding-window note: half the layers use sliding-window attention. The table computes KV with the standard full-attention formula; real long-context usage is lower, so verdicts are conservative.
VRAM by Quant Tier (@8K context, fp16 KV)
| Quant | Weights | Total (incl. 1.5GB runtime overhead) |
|---|---|---|
| Q8_0 | 22.2 GB | 24.1 GB |
| Q6_K | 17.2 GB | 19.1 GB |
| Q4_K_M | 12.8 GB | 14.7 GB |
| Q3_K_M | 10.4 GB | 12.4 GB |
| Q2_K | 8.3 GB | 10.2 GB |
KV cache and runtime overhead do not change across quants; longer contexts grow the KV part linearly — adjust context length in the GPU checker tool.
GPU Verdict Matrix (Q4_K_M @8K)
| GPU | Usable VRAM | Verdict | Recommended quant | Theoretical speed | |
|---|---|---|---|---|---|
| RTX 3060 12GB | 12.0 GB | Not feasible | Q2_K | ≈189 tok/s | Try it → |
| RTX 3090 | 24.0 GB | Comfortable | Q6_K | ≈237 tok/s | Try it → |
| RTX 4070 Ti Super | 16.0 GB | Tight fit | Q4_K_M | ≈229 tok/s | Try it → |
| RTX 4090 | 24.0 GB | Comfortable | Q6_K | ≈256 tok/s | Try it → |
| RTX 5090 | 32.0 GB | Comfortable | Q8_0 | ≈351 tok/s | Try it → |
| RTX A6000 | 48.0 GB | Comfortable | FP16 | ≈80 tok/s | Try it → |
| A100 80GB | 80.0 GB | Comfortable | FP16 | ≈212 tok/s | Try it → |
| H100 80GB | 80.0 GB | Comfortable | FP16 | ≈349 tok/s | Try it → |
| RX 7900 XTX | 24.0 GB | Comfortable | Q6_K | ≈244 tok/s | Try it → |
| Mac mini M4 Pro (48GB) | 36.0 GB | Comfortable | Q8_0 | ≈53 tok/s | Try it → |
| Mac Studio M4 Max (64GB) | 48.0 GB | Comfortable | FP16 | ≈57 tok/s | Try it → |
| Mac Studio M3 Ultra (96GB) | 72.0 GB | Comfortable | FP16 | ≈85 tok/s | Try it → |
Theoretical speed = bandwidth × 0.75 ÷ per-token weight bytes (active params for MoE); real-world results vary with framework/driver/CPU, ±30%.
FAQ
- Can a 16GB GPU run gpt-oss-20b?
- Tight fit. Q4_K_M @8K is ~14.7GB, leaving ~1.3GB on a 16GB card (RTX 4070 Ti Super); the tiny KV cache (~0.4GB) keeps long contexts cheap too.
- gpt-oss-20b or Qwen3 30B-A3B?
- On 16GB only gpt-oss-20b fits (Q4 ~14.7GB). On 24GB both work — pick gpt-oss for OpenAI-style reasoning and minimal KV, 30B-A3B for denser knowledge coverage and the Qwen ecosystem.
- What is gpt-oss-20b's reasoning effort?
- A chain-of-thought intensity switch (low/medium/high) in OpenAI's harmony format. Use low for simple tasks to save tokens, high for math and code — the biggest usage difference from ordinary chat models.
Related guides
- MoE Model Hardware Requirements: Total vs Active Parameters
- Why Memory Bandwidth Determines LLM Inference Speed
- Quantization Explained: Q4 vs Q8 and What You Actually Lose
Try gpt-oss-20b in the GPU compatibility checker →
Data verified 2026-09-01