Running DeepSeek-R1 Locally: The Complete Hardware Guide

updated 2026-08-04 · verified 2026-08-04

DeepSeek-R1 is the model that made everyone want local AI — and the model least suited to it. Here’s the complete picture, including the paths YouTube doesn’t emphasize.

Why R1 is different

DeepSeek-R1 is a 671B mixture-of-experts model with 256 routed experts, of which only 37B parameters activate per token. Two consequences:

• Capacity is brutal: all 671B parameters must be loaded. At Q4_K_M that’s ~411GB of weights — plus, mercifully, almost nothing for KV cache. R1 uses MLA (multi-head latent attention), which compresses KV cache to ~0.6GB at 8K context where a standard architecture would need tens of GB.
• Speed is surprisingly decent: because only 37B parameters are read per token, R1 generates at the speed of a ~37B model, not a 671B one — if you can hold it.

The realistic paths

Path 1: Don’t run the full model — use a distill (recommended for 95% of people)

DeepSeek released distills fine-tuned on R1’s reasoning traces. R1-Distill-Qwen-32B needs ~24GB at Q4_K_M — one used RTX 3090. It’s not the full R1, but it reproduces much of the reasoning style, and it’s the model those “R1 on my Mac” videos are usually running.

Path 2: Multi-GPU rig (the enthusiast path)

• 8× used RTX 3090 (192GB… no). Let’s be precise: Q4_K_M needs ~413GB total. That’s 14× 3090s, or more practically 6× A100 80GB / H100s — firmly datacenter territory.
• At Q2_K (~215GB): 8× 3090 (192GB) still short; you need ~9–10 cards or 3× A100 80GB.
• Reality check: this is why essentially nobody self-hosts full R1 on consumer hardware.

Path 3: CPU/ RAM inference (the patient path)

A used dual-Epyc workstation with 512GB–1TB of DDR4/DDR5 can hold R1 at Q4 entirely in system RAM. Speed: 3–6 tok/s on a good day — usable for batch jobs and “set it running overnight” tasks, not for chat.

Path 4: The API (the sane path for most)

DeepSeek’s own API has moved on to the V4 generation (V4-Flash at $0.14/$0.28 per million tokens — astonishingly cheap). Third parties still host R1 itself: Together AI at $3.00/$7.00, DeepInfra’s R1-0528 at $0.50/$2.15 per million tokens. Unless you’re processing billions of tokens monthly or have hard privacy requirements, the API wins on cost alone — see our cost calculator.

If you’re doing it anyway

1. Start with unsloth’s dynamic quants of R1 — their Q2_K_XL/Q3 mixes keep critical layers at higher precision and measurably beat naive Q2.
2. MLA means long contexts are cheap — R1’s 128K context costs you almost nothing extra in VRAM. Use it.
3. Set realistic expectations: even a 3× A100 setup lands in the ~15 tok/s range for generation.

FAQ

Sources

• deepseek-ai/DeepSeek-R1 (Hugging Face config)
• DeepSeek API pricing