How Fast Is Strix Halo, Really? Real Tokens-Per-Second for Local LLMs (Dense vs MoE)

Quick answer: On the AMD Ryzen AI Max+ 395 ("Strix Halo"), tokens-per-second depends far more on model architecture than model size. A dense 70B model at 4-bit generates only about 5 tok/s; a 30B Mixture-of-Experts model such as Qwen3-30B-A3B runs at 72–75 tok/s on the same machine; GPT-OSS 120B (also MoE) lands at 31.41 tok/s; and small 7–8B dense models measure 42–53 tok/s. That ~15× swing on identical silicon comes down to one number — memory bandwidth (~256 GB/s theoretical, ~215 GB/s real) — because token generation is bandwidth-bound and MoE models read far fewer weights per token. The takeaway: buy Strix Halo for the MoE models you'll actually run in 2026, not the dense 70B the spec sheet implies.

The one number that decides token speed: memory bandwidth

Almost everyone shopping a 128GB Strix Halo box gets the same thing wrong. They see "runs a 70B model" and assume the box is slow or fast as a single fact. It isn't. Once a model fits in memory — and 128GB fits a lot — the thing that caps how fast it generates tokens is memory bandwidth, not compute and not spare capacity.

The mechanism is simple enough to do on a napkin. To produce each new token, the box has to read the model weights it needs for that token out of memory. So, roughly:

generation speed ≈ memory bandwidth ÷ bytes read per token

The Ryzen AI Max+ 395 moves about 256 GB/s on paper and ~215 GB/s in the real world (per the box spec sheets and independent llama.cpp testing). That's roughly double a normal desktop APU — the reason it produces usable tokens at all — but it's the ceiling everything else runs into. That single pool is what unified memory buys you: capacity without a VRAM wall, at a bandwidth well under a discrete GPU's. For a background primer on the chip itself, see our Ryzen AI Max+ 395 explainer.

Here's how that bandwidth stacks up against the alternatives, because the number reframes the whole comparison:

So Strix Halo sits at the bottom of that bandwidth stack, close to the DGX Spark and about 40% of a Mac Studio M4 Max. That's the "local AI is slow" reputation in one line — and it's correct for dense models. It's also, as we're about to see, completely wrong for the models most people run in 2026.

Dense vs MoE: the 15× swing on identical silicon

This is the reveal, and it's the single most important thing a 2026 Strix Halo buyer needs to internalize. Two models can have similar capability and run at wildly different speeds on the exact same box, because of how many weights they read per token.

Dense models — Llama 3.3 70B, most classic models — activate every parameter for every token. A dense 70B at 4-bit is ~40GB of weights, and all of it streams through that ~215 GB/s pipe for each token produced. Do the division and you land in single digits: ~5 tok/s. Bandwidth-starved. This is the number the scary headlines quote.

Mixture-of-Experts (MoE) models flip the math. The full model sits resident in the 128GB pool, but only a fraction of it — a handful of "experts" — activates for any given token. Qwen3-30B-A3B is the poster child: 30B total parameters, but the "A3B" means only about 3B are active per token. So the per-token memory traffic is closer to a 3B model than a 30B one, and the same box that crawls on dense 70B flies at 72–75 tok/s.

"On the AMD Ryzen AI Max+ 395, a dense 70B model generates only about 5 tokens per second, but a 30B mixture-of-experts model such as Qwen3-30B-A3B runs at 72–75 tokens per second on the same machine — because token generation is capped by memory bandwidth (~256 GB/s), and MoE models read far fewer weights per token than dense models of similar size."

Same silicon. Same memory. Roughly 15× the speed, purely because of architecture. This is why "how fast is Strix Halo?" has no single answer — and why the 2026 answer is a lot more optimistic than the 2024 one, when dense models dominated. The MoE wave (Qwen3, GPT-OSS, DeepSeek, Mixtral descendants) is the best thing that ever happened to bandwidth-limited unified-memory boxes.

The real tokens-per-second table (by model, 4-bit)

Here's the artifact this post exists to provide: a clean, buyer-facing tokens-per-second table by model, with the dense-vs-MoE split front and center. Every figure is sourced or labeled — per our house rule, we do not invent tok/s numbers. Treat all figures as 4-bit quantization, realistic ranges (they're quant-, context-, and runtime-dependent), not guarantees.

Sources & caveats:

• 7–8B dense = 42–53 tok/s. The measured spread across GPU backends at pp512/tg128: Shisa V2 8B at 42.0 (HIP/ROCm), Llama 2 7B at 45.8 (Vulkan), 47.9 (HIP/ROCm), 50.88 (HIP+WMMA+FA) and 52.73 (Vulkan+FA) — Level1Techs and llm-tracker.info. The same chip with no iGPU offload does 28.94. We stop this row at 8B because no 13B-class run is published; the older "7–13B ≈ 30–45 tok/s" band that stood here was neither.
• 30B MoE = 72–75 tok/s. Qwen3-30B-A3B at UD-Q4_K_XL on Vulkan, pp512/tg128: 72.0 on Level1Techs, 75.32 on llm-tracker.info — two captures of the same config. No published run approaches 100, which is where our old top-of-band came from.
• GPT-OSS 120B = 31.41 tok/s. ServeTheHome's out-of-the-box LM Studio run on the Beelink GTR9 Pro, which shares the exact APU and memory of every other Strix Halo box. The same review measured the box at 125–128W and 39–41 dBA while that model generated (power and noise page). StorageReview independently ran GPT-OSS 120B on the HP Z2 Mini G1a with no discrete GPU, cross-checking the datapoint on a second machine.
• Dense 70B ≈ 5 tok/s. ServeTheHome reports "around 5 tokens/second" on Llama 3.3 70B; Shisa V2 70B measures 5.0 on Level1Techs. Two sources, same answer.
• Cut: a "Phi-3.5 up to 61 tok/s (AMD self-reported)" row used to sit in the table above. We could not find the AMD page it was attributed to, so it is gone rather than labeled.

Every figure we can trace to a published measurement now lives in our tokens/sec benchmark reference — one row per hardware × model × quantization × backend, each linking the page its number was read from, at the exact run spec the source used. The bands in the table above are wider than any single measurement; the reference page is where the individual sourced datapoints are, along with an explicit list of the segments where no verified benchmark exists.

For the deep dive on the 120B MoE case specifically — quant choices, which box, real setup — see how to run GPT-OSS 120B locally on a mini PC.

Prefill vs decode: the number benchmarks hide

Every tok/s figure above describes decode — the token-by-token generation speed after your prompt is processed. There's a second number that matters and that most benchmarks quietly omit: prefill, the time to process the input prompt before the first token appears.

Prefill is compute-bound, not bandwidth-bound, and on unified-memory boxes like Strix Halo it is generally slower than on a discrete GPU. For short chat prompts you'll never notice. But feed it a long context — a big RAG retrieval, a whole code file, a lengthy system prompt — and prefill adds real latency before generation starts. A box that decodes at 80 tok/s can still make you wait a beat on first token if the prompt is thousands of tokens long.

We won't fabricate a prefill tok/s figure — it varies too much with context length, batch, and runtime — but the honest expectation is: prefill on Strix Halo is generally slower than a dGPU, and it's the caveat that matters most for RAG and coding workloads. If your use is interactive chat with short prompts, ignore it. If it's long-context retrieval, factor in first-token latency on top of the decode numbers above.

Which box changes the number — and which doesn't

Here's a mistake that costs money: agonizing over which Strix Halo box is "fastest." For peak tokens/sec, they're effectively identical, because they run the same Ryzen AI Max+ 395 silicon and the same ~256 GB/s memory (the full field lives on our Strix Halo hub):

• GMKtec EVO-X2 — the flagship, quiet dual-M.2 appliance ($3,649)
• Framework Desktop — cheapest credible 128GB box, standard mini-ITX ($3,449)
• Beelink GTR9 Pro — dual 10GbE + vapor-chamber cooling ($4,349)
• Minisforum MS-S1 Max — best I/O + 320W PSU ($3,799)
• HP Z2 Mini G1a — business-grade PRO silicon, ECC, 3-yr warranty ($5,349–$7,406)

What actually differs between them is sustained throughput, not peak. Cooling and power delivery decide how long a box holds its peak tok/s before thermal throttling drags it down. The Beelink GTR9 Pro's vapor-chamber dual-fan cooler and the Minisforum MS-S1 Max's 320W PSU and oversized heatsink are built to sustain ~120W inference longer than a palm-sized box can (GTR9 Pro vs MS-S1 Max splits those two on cooling and I/O). ServeTheHome measured the GTR9 Pro at 125–128W and 39–41 dBA while it generated on GPT-OSS 120B, against 16–25W and 36–37 dBA idle (power and noise page) — while also flagging a 10GbE NIC crash under sustained iGPU load, a driver issue worth knowing.

The bottom line: peak tok/s is a silicon fact, identical across boxes; sustained tok/s is a cooling fact, where the better-built boxes win. If you'll run long, back-to-back inference sessions, the cooling matters. If you hit the box in bursts, any of them delivers the same peak — the practical split is build quality and I/O, which we lay out in EVO-X2 vs GTR9 Pro and EVO-X2 vs Framework Desktop.

Two adjacent platforms shift the number in opposite directions. The NVIDIA DGX Spark (273 GB/s, CUDA) is a hair faster on bandwidth and wins on software/clustering, not raw single-box speed — we break that down in DGX Spark vs Strix Halo, and spec-for-spec in DGX Spark vs EVO-X2; the rest of the GB10 field sits on our DGX Spark hub. And if you want genuinely higher token speed at 128GB, the Mac Studio M4 Max (up to 546 GB/s) is the move — M4 Max vs M3 Ultra covers that step up, and the Apple Silicon hub has the rest of the lineup.

Budget-tier reality: the 7–13B boxes

Not everyone needs 128GB. If you'll never touch a 30B+ model, a sub-$1,000 box runs 7–13B models for a fraction of the money — but be clear what that rests on: no published single-stream benchmark exists for any of these exact SKUs, so we quote no number for them. The 42–53 tok/s above is Strix Halo at ~215 GB/s; these boxes are slower and narrower, and extrapolating down from a faster chip is a guess, not a measurement (we log it as an explicit gap on the benchmark reference). The GMKtec M6 Ultra ($569, Zen 4, 32GB) and Beelink SER8 ($799–$939, Ryzen 7 8845HS, 32GB) are the value picks here — M6 Ultra vs SER8 is the head-to-head. Think Qwen3 8B, Gemma 3 12B, or GPT-OSS 20B at 4-bit. Their ceiling is real — 32GB of RAM and modest bandwidth lock them out of 30B+ models — but for small local models and always-on agents they're the right buy, and you're not paying for capacity you won't use. For the full tiered breakdown, see the best mini PC for local LLMs in 2026.

So is Strix Halo fast enough for you?

Now the decision framework, using the numbers above rather than a vibe:

The reframe that should change your purchase: the scary "5 tok/s" number only applies to dense 70B models — which, in 2026, fewer and fewer people actually run. The models that matter now are MoE, and on MoE the same ~$3,500 box is comfortably interactive. Buy Strix Halo for the workload you'll actually have, not the dense-70B strawman the spec sheet implies. If you run MoE — and you probably will — it's fast enough, and then some.

Still deciding which specific box, or whether a unified-memory machine beats a GPU build for your workload? Start with our GMKtec EVO-X2 review for the flagship pick, or the tiered buyer's guide for every budget.

Bottom line

"How fast is Strix Halo?" is the wrong question — the right one is "how fast is Strix Halo on the model I run?" On the Ryzen AI Max+ 395, a dense 70B crawls at ~5 tok/s while a 30B MoE model like Qwen3-30B-A3B flies at 72–75 tok/s and GPT-OSS 120B holds 31.41 tok/s — a 15× spread on identical silicon, all explained by memory bandwidth (~215 GB/s real) and how many weights each architecture reads per token. The box you pick barely moves peak speed; the model architecture moves it enormously. For 2026's MoE-first reality, Strix Halo is fast enough, and the cheapest credible 128GB entry now sits near $3,450. Just buy it knowing what it's actually good at.