GPU Compatibility Checker

How to use this checker

The tool works in two directions, switched at the top:

1. Model → GPUs.Pick a model, quantization level and context length, and the table lists every GPU with its verdict, theoretical speed and reference price (used price where available). GPUs that can't hold the model alone get a multi-card suggestion: the smallest viable count (2–4 identical cards), combined usable VRAM, total price and speed.
2. GPU → models.Pick a GPU and card count, and the table lists each model with its verdict, recommended quantization and theoretical speed. The recommendation picks the highest-quality level that still lands in comfortable or tight — a tight Q4 beats a comfortable Q2, because quality loss hurts more than a thin VRAM margin.
3. Treat speeds as upper bounds.tok/s is estimated as memory bandwidth × 0.75 ÷ weight bytes read per token. It is a theoretical estimate, ±30% depending on framework, drivers and CPU.

What the four verdicts mean

Multi-GPU and the cloud fallback

For identical cards, usable VRAM adds with no discount — 2×24GB is judged as 48GB. The cost is speed: combined bandwidth is multiplied by 0.85, and mixed cards are limited by the slowest one. Unified-memory Macs first have their nominal memory multiplied by 0.75 before the verdict. When neither single nor multi-card works, the page shows cloud GPU rental links: billed by the hour, sensible when you only need a big model occasionally and buying hardware doesn't pay off.

How MoE models are counted

Mixture-of-experts models use two different parameter counts: VRAM follows total parameters — every expert must be loaded, so Qwen3-30B-A3B needs memory for the full 30.5B; speed follows active parameters — each token reads only the 3.3B of activated weights, so its theoretical speed on an RTX 4090 resembles a dense 3B model, not a 30B one. A three-digit tok/s estimate next to a "30B total, 3B active" model is not a bug.

FAQ

Can an RTX 4090 run Llama 70B?

Not alone — Llama 3.3 70B at Q4_K_M needs about 47GB, nearly double the 24GB on a 4090. Two 4090s (48GB) run it tight at ~30 tok/s (theoretical estimate ±30%); a single RTX 5090 (32GB) also falls short.

What GPU do I need for a 32B model?

A 32B model at Q4_K_M needs about 24GB at 8K context — tight on an RTX 4090/3090, comfortable on an RTX 5090 or any 48GB card.

Is a Mac good for local LLMs?

Yes for capacity: unified memory lets a 96GB Mac Studio hold models no consumer GPU can. Speed is lower than NVIDIA per dollar — a Mac Studio M3 Ultra runs a 70B Q4 model at roughly 10-15 tok/s.

What if the verdict is "needs multi-GPU" or "infeasible"?

The tool shows the smallest viable 2-4 card setup of the same GPU: usable VRAM adds directly with no discount, and speed is estimated at 0.85× the combined bandwidth (theoretical estimate ±30%). If even four cards cannot hold the model, renting a cloud GPU by the hour is the practical fallback — the page links to rental options.

Further reading

• Best GPU for Local LLMs in 2026Every budget tier, from a used 3060 to Mac Studio
• Multi-GPU Setup for Local LLMsNVLink, PCIe and layer splitting explained
• MoE Model Hardware RequirementsVRAM counts total params, speed counts active params
• Mac vs GPU for LLM InferenceUnified memory explained