GPUFits
GPUs / RX 7900 XTX
RX 7900 XTX for Local LLMs: What It Runs and How Fast
The RX 7900 XTX is the budget shortcut to 24GB: 960 GB/s of bandwidth beats the 3090 on paper, at used prices around $600-700 — half of a same-capacity N-card. It runs 8B Q4 at a theoretical ~150 tok/s, on par with a 4090: llama.cpp's GGUF inference leans lightly on CUDA-specific features, so AMD's bandwidth utilization holds up.
The cost is ecosystem: ROCm works on Linux, llama.cpp has a Vulkan fallback, but Windows support, kernel optimizations in some frameworks, and multi-GPU support are all clearly weaker than CUDA — when things break, you read GitHub issues yourself. Profile: a Linux user who enjoys tinkering, is budget-sensitive, and targets the single-card 24B tier (24B Q4 comfortable, 32B tight — exactly the 3090's class). Do not buy it as a “first LLM card” for someone who won't tinker: the NVIDIA premium buys ecosystem certainty, and for many people that money is well spent.
Specs
| Nominal VRAM | 24 GB |
| Usable VRAM (for models) | 24.0 GB |
| Memory bandwidth | 960 GB/s |
| Type | Discrete GPU |
| Release year | 2022 |
| MSRP | $999 |
| Used reference price | $650 |
| TDP | 355 W |
Value Metrics
Based on used reference price; used prices are market estimates, see commentary for volatility.
Model Verdict Matrix (Q4_K_M @8K)
| Model | VRAM needed | Verdict | Recommended quant | Theoretical speed | |
|---|---|---|---|---|---|
| Llama 3.2 3B | 4.4 GB | Comfortable | FP16 | ≈112 tok/s | Try it → |
| Llama 3.1 8B | 7.5 GB | Comfortable | FP16 | ≈45 tok/s | Try it → |
| Qwen3 8B | 7.7 GB | Comfortable | FP16 | ≈44 tok/s | Try it → |
| Phi-4 14B | 12.2 GB | Comfortable | Q8_0 | ≈46 tok/s | Try it → |
| Mistral Small 3.2 24B | 17.5 GB | Comfortable | Q6_K | ≈37 tok/s | Try it → |
| Gemma 3 27B | 22.4 GB | Tight fit | Q4_K_M | ≈43 tok/s | Try it → |
| Qwen3.8 27B | 20.7 GB | Tight fit | Q5_K_M | ≈36 tok/s | Try it → |
| Muse Glimmer 30B | 20.1 GB | Tight fit | Q5_K_M | ≈34 tok/s | Try it → |
| Qwen3 30B-A3B | 21.0 GB | Tight fit | Q4_K_M | ≈356 tok/s | Try it → |
| Qwen3 32B | 23.7 GB | Tight fit | Q4_K_M | ≈36 tok/s | Try it → |
| gpt-oss-20b | 14.7 GB | Comfortable | Q6_K | ≈244 tok/s | Try it → |
| Llama 3.3 70B | 47.4 GB | Not feasible | needs 2 cards | — | Try it → |
| gpt-oss-120b | 73.6 GB | Not feasible | needs 4 cards | — | Try it → |
| DeepSeek-R1 671B | 413.1 GB | Not feasible | — | — | Try it → |
Theoretical speed = bandwidth × 0.75 ÷ per-token weight bytes (active params for MoE); real-world results vary with framework/driver/CPU, ±30%.
FAQ
- What can the RX 7900 XTX run?
- Identical VRAM math to the 3090 (24GB): 24B tier comfortable, Qwen3 32B tight (~23.7GB), 70B only via dual cards (not advised — weak AMD multi-GPU ecosystem). At 960 GB/s its theoretical speed slightly beats the 3090.
- How is llama.cpp on AMD cards?
- On Linux, both ROCm and Vulkan backends work and GGUF inference nears theoretical speed; Windows and some frameworks (e.g. vLLM) are a tier weaker. Tinkering ability is the prerequisite for turning $650 into 24GB.
- 7900 XTX or 3090?
- Linux + tinkering + budget: 7900 XTX — half the price, same 24GB, more bandwidth. Ecosystem certainty, full CUDA framework coverage, future multi-GPU: 3090.
Related guides
- Run GGUF Models Locally: llama.cpp and Ollama Walkthrough
- Best GPU for Local LLMs in 2026: Every Budget Tier
- Why Memory Bandwidth Determines LLM Inference Speed
Try RX 7900 XTX in the GPU compatibility checker →
Data verified 2026-09-01