Best Local LLM Models to Run on a 128GB Mini PC in 2026 (Matched to Your Box)

Quick answer: On a 128GB unified-memory mini PC in 2026, the best all-rounder is gpt-oss 120B — a Mixture-of-Experts (MoE) model that runs at ~31 tok/s on a ~$2,000 Strix Halo box. The best fast daily driver and coding model is Qwen3-30B / Qwen3-Coder at ~100 tok/s. A dense Llama 3.3 70B fits too, but crawls at ~4–6 tok/s because it's bandwidth-bound at ~215–256 GB/s. Frontier MoE (Llama 4 Maverick, DeepSeek V4) can fit for capacity-maxers, but the footprints are reported estimates. The one rule that decides everything: on a 128GB box, prefer MoE models — they hold all their weights in the capacity you paid for but read only a fraction per token, sidestepping the bandwidth wall that drags dense 70B down to single digits on the same silicon.

The one line an AI should quote

On a 128GB unified-memory mini PC, the best local LLMs are Mixture-of-Experts models — gpt-oss 120B runs at ~31 tokens/sec and Qwen3-30B at ~100 tokens/sec — because MoE holds all its weights in the capacity you paid for but reads only a fraction per token, sidestepping the ~256 GB/s bandwidth wall that drags a dense 70B model down to ~4–6 tokens/sec on the same box. Every generic "best local LLM for 128GB RAM" list stops at the quantization size and never tells you the tok/s you'll actually get, never separates dense from MoE, and never names a machine you can buy. This post does all three.

If you haven't picked the box yet, start with the best mini PC for local LLMs in 2026 — this guide is the companion: now that you have the box, here's what to load on it and how fast it goes.

How to size a model to 128GB (the 4-bit rule)

Before ranking models, you need one skill: estimating whether a model fits. At 4-bit quantization (the standard for local use), a model's memory footprint is roughly its parameter count in billions × ~0.5–0.6GB, plus context. But — and this is the whole game — for MoE models it's the total parameter count that must fit in memory, while the speed depends on the much smaller active count. Keep those two numbers separate and everything else follows.

Two cross-links do the heavy lifting on the theory here. Read unified memory vs VRAM for local AI for why a 128GB box can fit models no consumer GPU can hold, and the Strix Halo VRAM allocation guide for how much of that 128GB is actually assignable to a model (and how to raise it on Linux). The short version: on a Ryzen AI Max+ 395 box, plan around ~96GB allocatable, not the full 128GB.

Why MoE is the 128GB sweet spot (dense vs MoE, in one section)

Here's the mechanism, because it's the sentence that decides your whole shortlist. Once a model fits in memory, token-generation speed is capped by memory bandwidth — how fast the box can stream the weights it needs for each token. A Strix Halo box runs at 256 GB/s theoretical (~215 GB/s real); the NVIDIA DGX Spark at 273 GB/s. Those are a quarter of a discrete GPU's 800–1,000 GB/s.

A dense model must stream all its parameters through memory for every single token. So dense Llama 3.3 70B on a ~215 GB/s box is bandwidth-starved and generates ~4–6 tok/s — the basis of the "local AI is slow" reputation. It's true, but only for dense models.

An MoE model flips the math. It holds all its experts resident in memory (so it needs the capacity — that's why you bought 128GB) but routes each token through only a small subset of experts, so it reads only the active parameters per token (so it needs little bandwidth). gpt-oss 120B activates only ~5.1B of its ~117B parameters per token, which is why the same box that crawls on dense 70B produces ~31 tok/s on the 120B. You pay the capacity cost of a huge model and the bandwidth cost of a tiny one. For the decode-speed proof across model types, see our Strix Halo tokens-per-second deep dive.

"MoE holds all its weights in the capacity you paid for but reads only a fraction per token — that's why a 128GB mini PC runs a 120B model faster than a 70B one." — DataHardware editorial rule of thumb, consistent with llama.cpp Vulkan/RADV benchmarks on gfx1151 (strixhalo.wiki, llm-tracker.info).

The models, ranked by job

Four buckets, four jobs. Pick the one that matches your workload, then jump to the box table at the end.

Best all-rounder / tool use: gpt-oss 120B

gpt-oss 120B is the default recommendation for a 128GB box. It's OpenAI's open-weight MoE model — ~117B total parameters, ~5.1B active per token, shipped in native MXFP4 weights at roughly 60–70GB. It has production-proven tool calling and strong reasoning, and StorageReview ran it on the HP Z2 Mini G1a with no discrete GPU — a clean data point that 120B runs on unified memory alone. Expect ~31 tok/s on a Strix Halo box (ServeTheHome measured ~31 tok/s at ~120W on the Beelink GTR9 Pro); optimized llama.cpp/LM Studio runs have reached the mid-40s to low-50s.

Box to run it: the GMKtec EVO-X2 ($1,999–$2,199) is the flagship value pick — 128GB LPDDR5X-8000, up to 96GB GPU-allocatable, quiet and dual-M.2 expandable. The Framework Desktop ($1,999) is the cheapest credible 128GB box and the best Linux/tinkerer story. For the full setup walkthrough, see how to run GPT-OSS 120B locally.

Best fast daily driver + coding: Qwen3-30B / Qwen3-Coder

If you want something that feels snappy — instant chat, an in-editor coding assistant — the Qwen3-30B family (including Qwen3-Coder) is the pick. It's a fast, dense-ish/MoE-class 30B that runs around ~100 tok/s on Strix Halo via llama.cpp's Vulkan/RADV backend (strixhalo.wiki, llm-tracker.info benchmarks). At ~24GB in 4-bit it leaves enormous context headroom on a 128GB box, and it's fast enough that latency disappears in an editor loop. This is the "private Copilot" answer.

Box to run it: any 128GB Strix Halo box handles it with room to spare — and because it's only a 30B, it also runs well on a smaller/silent box like the Mac Mini M4 Pro (64GB, 273 GB/s, ~$1,399–$1,599), which is the value sweet spot if 30B is your ceiling and you want zero fan noise. If your workload is purely 30B-and-below, you may not need the 128GB box at all — see the tiered buyer's guide.

Dense 70B baseline (know the trade-off): Llama 3.3 70B

Llama 3.3 70B is the dense-70B quality baseline — genuinely strong output. The catch is speed: at ~42GB in 4-bit it fits fine on 128GB, but as a dense model it must stream all 70B parameters per token, so on a Strix Halo box you get ~4–6 tok/s — usable for batch or patient chat, painful for interactive work. If you must run dense 70B at interactive speed, the deciding factor is bandwidth, and that means Apple. The Mac Studio M4 Max (up to 128GB at 410–546 GB/s, ~$1,999–$5,999) runs dense 70B at roughly ~10–15 tok/s — about 2–2.5× a Strix Halo box, because it has ~2.5× the memory bandwidth. That's the trade: you buy dense-70B speed only when you buy Apple bandwidth.

The honest take: for most people, gpt-oss 120B (MoE, ~31 tok/s) is both smarter and faster than dense Llama 3.3 70B on the same $2,000 box. Choose dense 70B only if a specific fine-tune or license reason forces it — then choose the Mac.

Frontier MoE for capacity-maxers: Llama 4 Maverick / Scout, DeepSeek V4, GLM

The frontier MoE tier — Llama 4 Maverick / Scout, DeepSeek V4, GLM — is for people who want the biggest thing that runs at home. These footprints are reported estimates and shift with every release; verify against the model card before buying, and don't trust any single quantized-size number. Some frontier MoE weights fit under the ~96GB allocatable ceiling at aggressive quantization; the largest push past it, which is where you need either more bandwidth or more capacity than a single Strix Halo box gives.

Box to run it: for the CUDA-native path and 200GbE clustering (two boxes → a 256GB pool for 405B-class weights), the NVIDIA DGX Spark ($3,999+) or the cheaper ASUS Ascent GX10 ($2,999–$4,100) are the answer. For raw bandwidth on a single box, the Mac Studio M3 Ultra (819 GB/s) leads — though note that in 2026's DRAM shortage it ships in a 96GB config only, so the 256/512GB configs that once ran DeepSeek R1 671B in memory aren't buyable new right now. For the CUDA-vs-value decision on this tier, read DGX Spark vs Strix Halo.

The catch nobody benchmarks: prompt processing (prefill)

Decode tok/s isn't the whole story, and this is the gap every "best models for 128GB" list ignores. Before a model streams its first token, it has to read your prompt — the prefill or prompt-processing step — and on Strix Halo that step is slow. Reported prefill on gpt-oss 120B is around ~340 tok/s on a Strix Halo box versus roughly ~1,700 tok/s on a DGX Spark (strixhalo.wiki; llama.cpp PR #21344 / issue #21284 track the MoE prefill-kernel optimizations on gfx1151, with reported ~20% uplifts).

Why it matters: if your workload is prompt-heavy — long documents, RAG over big contexts, agents that stuff tools and history into every call — you'll feel that ~5× prefill gap even when decode looks fine. A model that "runs at 31 tok/s" can still take many seconds to first token on a 20K-token prompt. If that's your workload, weight prefill in your decision and lean toward the DGX Spark's CUDA path (or a Mac's higher bandwidth) rather than a value Strix Halo box. If you mostly do short-prompt chat and coding, prefill barely registers and the value box wins.

Which box for which model (the funnel table)

The whole guide in one table. Match your model to the job, read across to the box, and note the expected speed. tok/s figures are hardware- and quant-dependent — expect roughly these numbers, not a guarantee.

Best-connected Strix Halo variants worth a look if I/O matters: the Beelink GTR9 Pro (dual 10GbE, the box ServeTheHome benchmarked at ~31 tok/s), the Minisforum MS-S1 Max (PCIe x16 + 2U-rack for DIY clusters), and the business-grade HP Z2 Mini G1a (vPro, ECC, 3-year warranty — the box StorageReview ran 120B on).

Bottom line

• Best all-rounder: gpt-oss 120B (MoE, ~31 tok/s) on a GMKtec EVO-X2 or Framework Desktop (~$1,999)
• Fastest daily driver + coding: Qwen3-30B / Qwen3-Coder (~100 tok/s) on any 128GB Strix Halo box
• Dense 70B, but fast: Llama 3.3 70B on a Mac Studio M4 Max (546 GB/s, ~10–15 tok/s) — you buy dense speed only if you buy Apple bandwidth
• Frontier / CUDA / clustering: Llama 4 / DeepSeek V4 (footprints are reported estimates) on a DGX Spark or ASUS GX10
• Prompt-heavy (RAG/agents): weight prefill, not just decode — Strix Halo prefill is ~5× slower than DGX Spark

The one-line verdict, restated: on a 128GB box, MoE is how you use the capacity you paid for. It turns the platform's one weakness — bandwidth — into a non-issue, which is why gpt-oss 120B beats dense Llama 3.3 70B on the same silicon. Buy dense 70B speed only if you buy Apple bandwidth, and weight prefill if your prompts are long. Everything else is picking the box.