Hardware for Running Llama 3.1 405B Locally
Meta's largest dense model — 405B parameters that exceed any local box's memory even at Q4. Reach for Llama 3.3 70B locally instead.
Quick answer
Llama 3.1 405B (405B params) needs about 243 GB of memory even at Q4 — more than any single unified-memory box we track can allocate. You can't run it locally on one box; run a smaller distill or more aggressively quantized model instead — see the full model list. These are estimates including runtime overhead.
Estimated Memory Requirements
Estimated from the 405B parameter count (quantization × bytes/param + ~20% runtime overhead), not measured. MoE models load by total parameters.
| Precision | Est. Memory | Notes |
|---|---|---|
| FP16 (full precision) | ~972 GB | Best quality, highest memory usage |
| Q8 (8-bit quantized) | ~486 GB | Near-lossless quality, good balance |
| Q4 (4-bit quantized) | ~243 GB | Smallest footprint, slight quality loss |
No single box runs this locally
At 243 GB (estimated, Q4) Llama 3.1 405B exceeds the memory every unified-memory box in our catalog can allocate. Rather than list hardware that can't run it, we point you to models that can — browse the full model list for a distill or smaller sibling, or size any parameter count against the catalog with the VRAM calculator.
Disclosure: Some links on this page are affiliate links. We may earn a commission if you make a purchase — at no extra cost to you. This helps support our independent reviews.