GPUFits
Mac vs GPU for LLM Inference: Unified Memory Explained
updated 2026-09-10 · verified 2026-08-04
Apple Silicon is the odd one out in local LLMs: no VRAM, no CUDA, no fans you can hear — and it can hold models that would take four NVIDIA cards. This guide doesn’t take sides. It runs both paths through the same VRAM and speed formulas our tools use, then tells you who should buy which.
How unified memory changes the game
On a PC, the GPU has its own VRAM (24GB on a 4090) and the CPU has separate RAM. Models must fit in VRAM entirely, or you pay a brutal speed penalty copying layers over PCIe: once you partially offload to system RAM, effective bandwidth collapses to the 60–100 GB/s range, and an 8B model drops from ~150 tok/s on a 4090 to ~12 tok/s (see is CPU offload worth it).
Apple Silicon puts one memory pool on the chip package, shared by CPU and GPU. A 96GB Mac Studio can hold a 70GB-class model in its entirety — no consumer NVIDIA card comes within 3× of that. “No copying” is a real structural advantage, but it solves a capacity problem, not a speed problem: the shared pool is LPDDR5, not GDDR6X/GDDR7, so bandwidth tops out lower — and it’s soldered to the package, so the number you buy is the number you keep.
The trade in one sentence: NVIDIA trades capacity for speed; Apple trades speed for capacity.
The bandwidth ladder (all verified)
| Chip | Bandwidth | Max memory | Rough 8B Q4 speed |
|---|---|---|---|
| M4 | 120 GB/s | 32GB | ~15 tok/s |
| M4 Pro | 273 GB/s | 64GB | ~40 tok/s |
| M4 Max (16-core) | 546 GB/s | 128GB | ~80 tok/s |
| M3 Ultra | 819 GB/s | 96GB¹ | ~120 tok/s |
| RTX 4090 (GDDR6X) | 1,008 GB/s | 24GB | ~150 tok/s |
| RTX 5090 (GDDR7) | 1,792 GB/s | 32GB | ~265 tok/s |
¹ Apple discontinued the 256GB/512GB Mac Studio options in 2026 due to the DRAM shortage; 96GB is the current ceiling, and the base price moved from $3,999 to $5,299.
The speed column uses the same formula as our tools: theoretical tok/s ≈ bandwidth × 0.75 (efficiency) ÷ weight bytes read per token. An 8B model at Q4_K_M has 8.03 × 4.9 ÷ 8 ≈ 4.9GB of weights, so on an M4 Pro: 273 × 0.75 ÷ 4.9 ≈ 42 tok/s. It’s a theoretical estimate — real numbers swing ±30% with framework, drivers, and thermals — but as a common yardstick it’s fair, and it’s good enough to eliminate wrong answers before you spend money.
The 75% rule
macOS doesn’t let the GPU have everything. It reserves memory for the system and display services, and the practical ceiling for GPU allocation is about 65–75% of physical RAM. Our tools model this as:
| Advertised memory | Usable VRAM (×0.75) |
|---|---|
| 32GB | 24GB |
| 48GB | 36GB |
| 64GB | 48GB |
| 96GB | 72GB |
The community workaround is sysctl iogpu.wired_limit_mb, which raises the cap. Two caveats: you’re borrowing from the OS itself (open a few browser tabs and you may hit swap), and bandwidth doesn’t increase by a single GB/s. Shop by the right-hand column, not the number on the box.
Work it yourself: four concrete examples
The formula: total requirement = weights (params × bpw ÷ 8) + KV cache + 1.5GB runtime overhead. Everything below assumes Q4_K_M (bpw = 4.9), 8K context, and fp16 KV — the same inputs as our GPU checker.
Example 1: Llama 3.1 8B → 4.9GB weights + 1.1GB KV + 1.5GB overhead ≈ 7.5GB. Any 12GB card or 32GB+ Mac handles it comfortably. ~40 tok/s on an M4 Pro Mac mini, ~150 tok/s on a 4090 — both far above reading speed. At this size the experience gap is small; buy whichever is cheaper.
Example 2: Qwen3-32B → 20.1GB weights + 2.1GB KV + 1.5GB ≈ 23.7GB. A 24GB 3090/4090 is tight — push the context to 32K and the KV cache grows to 8.4GB, and it no longer fits. A 48GB Mac mini (36GB usable) holds it comfortably but generates ~10 tok/s, versus ~38 tok/s on the 4090. Same price, 3–4× faster on NVIDIA — that’s why ≤32B models belong on a GPU.
Example 3: Llama 3.3 70B → 43.2GB weights + 2.6GB KV + 1.5GB ≈ 47.4GB. A single 24GB card is out (you’d need two); the 48GB Mac mini (36GB usable) can’t hold it either; the 64GB Mac Studio (48GB usable) just barely makes it; only the 96GB Studio (72GB usable) is comfortable. Speed: ~14 tok/s on M3 Ultra, ~9–10 tok/s on M4 Max.
Example 4: gpt-oss-120b (MoE) → 71.5GB weights + 0.6GB KV + 1.5GB ≈ 73.6GB — note this exceeds the 96GB Studio’s 72GB usable pool, so the GGUF Q4_K_M build doesn’t fit; you need Q3_K_M (~60.5GB) or the official MXFP4 build. But MoE generation speed scales with active parameters (5.1B), not total: a theoretical ~200 tok/s on the M3 Ultra [estimate]. Huge capacity plus genuinely fast generation — this is exactly the Mac’s sweet spot.
Why Macs are slower: prefill and decode are different games
Token generation (decode) is a bandwidth game: every token re-reads the (active) weights once, which is what the formula above describes. But ingesting your prompt (prefill) is a compute game — and that’s Apple’s weak spot, since its GPUs have a fraction of the matrix throughput of same-price NVIDIA cards.
In practice: in chat you perceive decode speed, and the Mac holds up. But ask it to summarize a long document, or run an agent that re-prefills tens of thousands of tokens of context every turn, and the wait compounds. High-throughput, multi-user serving isn’t in the conversation at all — that’s CUDA and vLLM territory.
The money question: bandwidth per dollar
| Machine | Price | Bandwidth | Bandwidth per $ |
|---|---|---|---|
| Used RTX 3090 | $1,100 | 936 GB/s | 0.85 |
| Used RTX 4090 | $2,400 | 1,008 GB/s | 0.42 |
| Mac mini M4 Pro 48GB | $1,799 | 273 GB/s | 0.15 |
| Mac Studio M4 Max 64GB | $2,899 | 546 GB/s | 0.19 |
| Mac Studio M3 Ultra 96GB | $5,299 | 819 GB/s | 0.15 |
(Prices from our dataset’s 2026-08 snapshot; used prices are eBay-sold estimates.) On raw generation speed, a used NVIDIA card delivers 3–5× the bandwidth per dollar. What the Apple premium buys instead: capacity (72GB usable vs 24GB on one card), power and noise (50–100W for the whole machine vs 350–575W for a single card — run the long-term electricity difference through the cost comparator), and zero setup (Ollama / LM Studio / MLX, no drivers, no CUDA version hell).
Who should buy what
Buy a Mac if: you want a silent, sip-power box that holds huge models (70B at Q6_K, 120B-class MoE at lower quants), you value “download and run,” and your main use is solo chat and writing at ~10–14 tok/s on 70B — fine for chat, painful for agents.
Buy NVIDIA if: speed per dollar matters, you run CUDA tooling (vLLM, fine-tuning, Stable Diffusion), your models fit in 24–32GB anyway, or you need concurrency. A used 3090 at ~$1,100 embarrasses everything Apple makes on price-performance for ≤32B models — and when you outgrow it you can add a second card, while the Mac only scales by buying a whole new machine.
Common misconceptions
- “More memory means faster.” Capacity decides whether a model runs; bandwidth decides how fast. The 96GB Studio holds a 70B model, but at ~14 tok/s it’s still slower than any NVIDIA setup that can hold it.
- “A 96GB Mac runs 70B at Q8.” The math kills this one: Q8_0 weights alone are 75.1GB, and ~79GB with KV and overhead — over the 72GB usable ceiling. The sweet spot for 70B on a 96GB Mac is Q6_K (~62GB at 8K context).
- “If it doesn’t fit, just offload to RAM.” “Runs” and “usable” are different things: offloading drags effective bandwidth into a 60–100 GB/s cliff — 8B falls from ~150 to ~12 tok/s, and 70B becomes an exercise in patience.
- “Unified memory has no bottleneck.” Skipping the copy is real, but CPU and GPU share one pipe, and the OS and your other apps draw from the same 273–819 GB/s. It’s a wider highway, not an infinite one.
The MLX factor
Apple’s MLX framework is typically 10–20% faster than llama.cpp on the same Mac for LLM inference, and it’s now the default backend in several Mac frontends. If you go the Mac route, prefer MLX builds of your model (the mlx-community org on Hugging Face covers most mainstream models) — note that MLX uses its own quantization formats rather than GGUF, so the bpw numbers in our quantization table are only an approximation there.
A decision flow before you buy
- Pick the model and quant first, not the hardware. Start at Q4_K_M; move up a quant only if you have headroom.
- Do the VRAM math: weights + KV + 1.5GB overhead — and for a Mac, remember usable memory is 0.75× the sticker. Or just run the GPU checker.
- Do the speed math: bandwidth × 0.75 ÷ per-token weights. Below ~10 tok/s, chat starts to feel sluggish; if you’re building agents, tilt the budget toward bandwidth.
- Compare prices: compute bandwidth per dollar from the table above, then decide how much silence, low power, and zero maintenance are worth to you in Apple tax.
FAQ
Can a Mac mini run LLMs well?
The M4 Pro Mac mini (48GB, 273 GB/s) is a great quiet box for models up to 32B at Q4 — expect ~40 tok/s on 8B models and ~10 tok/s on 32B. Its real sweet spot is MoE: a model like Qwen3-30B-A3B with only 3.3B active params runs at a theoretical ~100 tok/s [estimate]. The base M4 (120 GB/s) works for 7B–8B models but feels slow on anything bigger.
Why can't my Mac use all its RAM for the GPU?
macOS reserves memory for the OS and display services, capping GPU allocation at roughly 65–75% of physical RAM. We model this as a 0.75 usable ratio — a 48GB Mac has ~36GB of usable VRAM, a 96GB machine ~72GB. You can raise the cap with sysctl iogpu.wired_limit_mb, but you're borrowing from the OS, and bandwidth doesn't increase by a single GB/s.
Is a Mac Studio M3 Ultra faster than an RTX 4090 for LLMs?
No for speed, yes for capacity. The M3 Ultra's 819 GB/s is below the 4090's 1,008 GB/s, so a 70B Q4 model generates ~14 tok/s on the Ultra — while the 4090 can't load that model at all (~47GB required). The 96GB Studio (72GB usable) holds 70B at Q6_K (~62GB) and gpt-oss-120b at Q3_K_M (~61GB), which no single 24GB card can touch.
Is a Mac good for agents or batch inference?
Not really. Decode speed is set by bandwidth, and ~14 tok/s on 70B is fine for solo chat. But agents re-ingest long contexts every turn, and prefill is compute-bound — exactly where Apple GPUs are weakest. For high-throughput or multi-user serving, NVIDIA with vLLM is far more efficient.
How much memory should my Mac have for LLMs?
Work backwards from the 0.75 ratio: 48GB (36GB usable) covers anything up to 32B at Q4; 64GB (48GB usable) barely squeezes a 70B Q4; 96GB (72GB usable) is where 70B Q6_K and 120B-class MoE models at lower quants become comfortable. Apple silicon memory is soldered — you can't upgrade later, so err on the larger side.