GPUFits
Models / gpt-oss-120b
gpt-oss-120b VRAM Requirements & GPU Pairing
gpt-oss-120b pushes MoE to the edge of local feasibility: 116.8B total parameters (128 experts, 4 active per token) with only 5.1B active, Apache-2.0. VRAM follows the total: Q4_K_M @8K is ~73.6GB — only A100/H100 80GB hold it single-card, tightly. The Mac Studio M3 Ultra 96GB (72GB usable) falls just short, but Q3_K_M (~60GB) is a workable tight fit — the most realistic home path.
Speed follows the 5.1B active params: a theoretical ~200 tok/s on M3 Ultra, an order of magnitude faster than dense 70B — MoE's true power at the large end. Sliding-window attention on half the layers plus headDim 64 keeps KV tiny (~0.6GB at 8K), so 128K context is painless. Against DeepSeek-R1: both are strong MoE reasoners, but gpt-oss-120b needs ~1/6 the VRAM — the strongest reasoning a consumer ceiling can touch. Full-blood R1 belongs to datacenters.
Architecture Specs
| Total parameters | 116.8B |
| Active parameters(MoE: VRAM follows total params, speed follows active params) | 5.1B |
| Layers | 36 |
| KV heads | 8 |
| Head dim | 64 |
| Native context | 128K |
| License | apache-2.0 |
| KV bytes per token (fp16) | 72.0 KB |
VRAM by Quant Tier (@8K context, fp16 KV)
| Quant | Weights | Total (incl. 1.5GB runtime overhead) |
|---|---|---|
| Q8_0 | 124.2 GB | 126.3 GB |
| Q6_K | 95.9 GB | 98.0 GB |
| Q4_K_M | 71.5 GB | 73.6 GB |
| Q3_K_M | 58.4 GB | 60.5 GB |
| Q2_K | 46.3 GB | 48.4 GB |
KV cache and runtime overhead do not change across quants; longer contexts grow the KV part linearly — adjust context length in the GPU checker tool.
GPU Verdict Matrix (Q4_K_M @8K)
| GPU | Usable VRAM | Verdict | Recommended quant | Theoretical speed | |
|---|---|---|---|---|---|
| RTX 3060 12GB | 12.0 GB | Not feasible | — | — | Try it → |
| RTX 3090 | 24.0 GB | Not feasible | needs 4 cards | — | Try it → |
| RTX 4070 Ti Super | 16.0 GB | Not feasible | — | — | Try it → |
| RTX 4090 | 24.0 GB | Not feasible | needs 4 cards | — | Try it → |
| RTX 5090 | 32.0 GB | Not feasible | needs 3 cards | — | Try it → |
| RTX A6000 | 48.0 GB | Not feasible | needs 2 cards | — | Try it → |
| A100 80GB | 80.0 GB | Tight fit | Q4_K_M | ≈490 tok/s | Try it → |
| H100 80GB | 80.0 GB | Tight fit | Q4_K_M | ≈804 tok/s | Try it → |
| RX 7900 XTX | 24.0 GB | Not feasible | needs 4 cards | — | Try it → |
| Mac mini M4 Pro (48GB) | 36.0 GB | Not feasible | needs 3 cards | — | Try it → |
| Mac Studio M4 Max (64GB) | 48.0 GB | Not feasible | needs 2 cards | — | Try it → |
| Mac Studio M3 Ultra (96GB) | 72.0 GB | Needs multi-GPU | Q3_K_M | ≈241 tok/s | Try it → |
Theoretical speed = bandwidth × 0.75 ÷ per-token weight bytes (active params for MoE); real-world results vary with framework/driver/CPU, ±30%.
FAQ
- What hardware can run gpt-oss-120b?
- Q4_K_M is ~73.6GB: A100/H100 80GB (tight); Mac Studio M3 Ultra (72GB usable) needs Q3_K_M (~60GB, tight); or 2×48GB A6000 multi-GPU. All consumer single cards are out.
- How fast is gpt-oss-120b?
- Only 5.1B active params: theoretically ~200 tok/s on M3 Ultra, ~490 on A100 — an order of magnitude faster than dense 70B, the extreme case of “VRAM by total, speed by active”.
- gpt-oss-120b or DeepSeek-R1?
- For local hardware, gpt-oss-120b: 60-74GB at Q3/Q4, reachable by Mac Studio or dual cards. Full R1 at Q4 is ~413.6GB — locally feasible only with datacenter multi-GPU or 1.58-bit dynamic quants, whose quality loss you must evaluate yourself.
Related guides
- MoE Model Hardware Requirements: Total vs Active Parameters
- Mac vs GPU for LLM Inference: Unified Memory Explained
- Why Memory Bandwidth Determines LLM Inference Speed
Try gpt-oss-120b in the GPU compatibility checker →
Data verified 2026-09-01