Methodology: how every number is computed

Nothing on this site is guessed or benchmarked on hardware we happen to own. Every VRAM figure, speed estimate, and verdict is the output of a fixed set of formulas applied to verified input data. This page publishes those formulas, their parameter sources, and their assumptions, so any number you see can be reproduced by hand. The implementations live in src/lib/vram.ts and src/lib/speed.ts; what the pages show is exactly what the code computes.

VRAM requirements: three additive parts

Total VRAM required = weights + KV cache + fixed overhead. All math uses GB (10⁹ bytes), not GiB.

1. Weights

weights GB = parameters (billions) × bpw ÷ 8, where bpw is bits per weight. MoE models use total parameters — every expert must be resident in memory even though only a subset fires per token. Our bpw values are not llama.cpp's theoretical bit-rates; they are calibrated from measured GGUF file sizes. Q4_K_M, for example, is 4.83 in theory but 4.9 in our table, back-computed from the actual byte counts of the Llama 3.1 8B GGUF files (embedding and metadata overhead included; small models deviate more from theory, large models sit closer).

2. KV cache

For standard GQA architectures (the vast majority of models):

KV cache GB = 2 × layers × KV heads × headDim × bytes per element × context tokens ÷ 10⁹

The factor of 2 is K and V; bytes per element defaults to 2 (fp16 KV) and drops to 1 with q8 KV quantization. DeepSeek's MLA architecture uses a separate branch — layers × (kvLoraRank + qkRopeHeadDim) × bytes × context — because MLA caches a compressed latent vector instead of full K/V. KV cache grows linearly with context length: the same model can need an order of magnitude more memory at 128K than at 8K, which is why every combination page on this site states its context length.

3. Fixed overhead

A flat 1.5 GB covers the CUDA context, runtime, and framework buffers. This is an estimated constant (a config value), not a measurement.

Worked example: Llama 3.1 8B @ Q4_K_M, 8K context

Architecture constants from the HF config.json: 8.03B parameters, 32 layers, 8 KV heads, headDim 128.

Weights = 8.03 × 4.9 ÷ 8 ≈ 4.9 GB
KV cache = 2 × 32 × 8 × 128 × 2 × 8192 ÷ 10⁹ ≈ 1.1 GB
Fixed overhead = 1.5 GB
Total ≈ 7.5 GB

That makes an 8GB card "tight" for 8B Q4_K_M at 8K (thresholds below), while a 12GB card is comfortable. Stretch the context to 128K and the KV cache alone becomes roughly 17 GB — a completely different answer.

Verdict thresholds

First, "usable VRAM": discrete GPUs use their nominal capacity (a 12GB card counts as 12GB); Apple unified memory is derated to 75% of nominal, because macOS only exposes roughly that fraction to the GPU. For multi-GPU setups, usable memory is the straight sum of the cards, with no discount. The verdict compares total requirement against usable capacity:

✅ Comfortable: requirement ≤ usable × 80%. Headroom for longer contexts or concurrent work.
⚠️ Tight: requirement ≤ usable × 100%. It fits, but with little margin — a longer context or OS memory pressure can push it into OOM.
🔀 Needs multi-GPU: usable × 100% < requirement ≤ usable × 120%. One card cannot hold it, but a second card usually closes the gap.
❌ Infeasible: requirement > usable × 120%. Needs multiple cards, a heavier quantization, or a shorter context.

The recommended quantization is the highest-quality tier that fits (comfortable or tight), scanning from Q8_0 down to Q2_K. A tight Q4 beats a comfortable Q2 — quality loss from quantization hurts more than thin VRAM headroom.

Speed estimation: bandwidth divided by bytes per token

Decoding reads the relevant weights once per generated token, so inference speed is dominated by memory bandwidth:

tok/s ≈ bandwidth (GB/s) × 0.75 ÷ bytes read per token (GB)

Bytes per token: dense models read their full weights (params × bpw ÷ 8); MoE models read only the active-parameter portion — the fundamental reason MoE inference is fast. The 0.75 efficiency factor (estimated) absorbs bandwidth-utilization losses. Two adjustments: identical multi-GPU setups get an effective bandwidth of Σbandwidth × 0.85 (inter-card communication cost); partial CPU offload caps effective bandwidth at min(VRAM bandwidth, system memory bandwidth), with system memory taken at 80 GB/s — this is the speed cliff.

Every speed figure is a theoretical estimate with a ±30% error band. Real throughput depends on the framework (llama.cpp / vLLM / Ollama), driver version, batch size, CPU, and thermals. These numbers reliably rank cards against each other; they are not a promise that you will hit X tok/s.

Pricing methodology

MSRP is the official launch price. Used prices are maintained only for cards released at least two years ago; they are dated estimate snapshots sampled from third-party trackers of eBay sold/listed prices (buysellram, gpupoet, gpudojo — US market), not live quotes. Sampling ranges are recorded in each card's _note field; a value of 0 means MSRP only. Prices are refreshed monthly.

Data sources and verification cadence

Model architecture parameters (layers, KV heads, headDim, context window) come from the raw config.json of each Hugging Face repository, archived under research/hf-configs/. Context windows use the official native length, not YaRN-extrapolated values.
Quantization bpw: back-computed from measured GGUF file sizes (HF x-linked-size response headers); calibration files and dates are in the _meta field of quant-presets.json.
GPU specs (VRAM, bandwidth): official NVIDIA / AMD / Apple specification pages, cross-checked against TechPowerUp.

All datasets are re-verified on the first weekend of every month. The current verification date appears in the footer and in site-config.json (verifiedAt); every change is logged on the data updates page.

Limitations

These are theoretical values, not measurements. We do not run every model on every card. The VRAM math matches reality closely (weights and KV cache are deterministic), but runtimes differ by ±5–10%; speed is a ±30%-class estimate.
Sliding-window and hybrid architectures are conservatively overestimated. For models with sliding-window attention (e.g. Mistral) or hybrid layer schemes, we compute KV cache as if every layer kept the full context — real requirements are lower.
No concurrency or batching. All figures assume a single interactive request at batch=1.
Not purchasing advice. See the disclaimer. If a number contradicts your measurements, tell us via the contact page.