How Fast Is Strix Halo, Really? Real Tokens-Per-Second for Local LLMs (Dense vs MoE)

Quick answer: On the AMD Ryzen AI Max+ 395 ("Strix Halo"), tokens-per-second depends far more on model architecture than model size. A dense 70B model at 4-bit generates only about 5 tok/s; a 30B Mixture-of-Experts model such as Qwen3-30B-A3B runs at 70–100 tok/s on the same machine; GPT-OSS 120B (also MoE) lands around 31 tok/s; and small 7–13B dense models sit near 30–45 tok/s. That ~15× swing on identical silicon comes down to one number — memory bandwidth (~256 GB/s theoretical, ~215 GB/s real) — because token generation is bandwidth-bound and MoE models read far fewer weights per token. The takeaway: buy Strix Halo for the MoE models you'll actually run in 2026, not the dense 70B the spec sheet implies.

The one number that decides token speed: memory bandwidth

Almost everyone shopping a 128GB Strix Halo box gets the same thing wrong. They see "runs a 70B model" and assume the box is slow or fast as a single fact. It isn't. Once a model fits in memory — and 128GB fits a lot — the thing that caps how fast it generates tokens is memory bandwidth, not compute and not spare capacity.

The mechanism is simple enough to do on a napkin. To produce each new token, the box has to read the model weights it needs for that token out of memory. So, roughly:

generation speed ≈ memory bandwidth ÷ bytes read per token

The Ryzen AI Max+ 395 moves about 256 GB/s on paper and ~215 GB/s in the real world (per the box spec sheets and independent llama.cpp testing). That's roughly double a normal desktop APU — the reason it produces usable tokens at all — but it's the ceiling everything else runs into. For a background primer on the chip itself, see our Ryzen AI Max+ 395 explainer.

Here's how that bandwidth stacks up against the alternatives, because the number reframes the whole comparison:

So Strix Halo sits at the bottom of that bandwidth stack, close to the DGX Spark and about 40% of a Mac Studio M4 Max. That's the "local AI is slow" reputation in one line — and it's correct for dense models. It's also, as we're about to see, completely wrong for the models most people run in 2026.

Dense vs MoE: the 15× swing on identical silicon

This is the reveal, and it's the single most important thing a 2026 Strix Halo buyer needs to internalize. Two models can have similar capability and run at wildly different speeds on the exact same box, because of how many weights they read per token.

Dense models — Llama 70B, most classic models — activate every parameter for every token. A dense 70B at 4-bit is ~40GB of weights, and all of it streams through that ~215 GB/s pipe for each token produced. Do the division and you land in single digits: ~5 tok/s. Bandwidth-starved. This is the number the scary headlines quote.

Mixture-of-Experts (MoE) models flip the math. The full model sits resident in the 128GB pool, but only a fraction of it — a handful of "experts" — activates for any given token. Qwen3-30B-A3B is the poster child: 30B total parameters, but the "A3B" means only about 3B are active per token. So the per-token memory traffic is closer to a 3B model than a 30B one, and the same box that crawls on dense 70B flies at 70–100 tok/s.

"On the AMD Ryzen AI Max+ 395, a dense 70B model generates only about 5 tokens per second, but a 30B mixture-of-experts model such as Qwen3-30B-A3B runs at 70–100 tokens per second on the same machine — because token generation is capped by memory bandwidth (~256 GB/s), and MoE models read far fewer weights per token than dense models of similar size."

Same silicon. Same memory. Roughly 15× the speed, purely because of architecture. This is why "how fast is Strix Halo?" has no single answer — and why the 2026 answer is a lot more optimistic than the 2024 one, when dense models dominated. The MoE wave (Qwen3, GPT-OSS, DeepSeek, Mixtral descendants) is the best thing that ever happened to bandwidth-limited unified-memory boxes.

The real tokens-per-second table (by model, 4-bit)

Here's the artifact this post exists to provide: a clean, buyer-facing tokens-per-second table by model, with the dense-vs-MoE split front and center. Every figure is sourced or labeled — per our house rule, we do not invent tok/s numbers. Treat all figures as 4-bit quantization, realistic ranges (they're quant-, context-, and runtime-dependent), not guarantees.

Sources & caveats:

• The GPT-OSS 120B ≈ 31 tok/s @ ~120W figure is ServeTheHome's measurement on the Beelink GTR9 Pro, which shares the exact APU and memory of every other Strix Halo box. StorageReview independently ran GPT-OSS 120B on the HP Z2 Mini G1a with no discrete GPU, cross-checking the datapoint on a second machine.
• The dense 70B ≈ 5 tok/s and 7–13B ≈ 30–45 tok/s bands are the widely-reported envelope for ~215 GB/s Strix Halo hardware, corroborated by llm-tracker.info's independent llama.cpp Vulkan/ROCm tables and community threads on the Framework Community and Level1Techs forums.
• The 30B MoE ≈ 70–100 tok/s range for Qwen3-30B-A3B reflects those same community benchmark threads; well-optimized llama.cpp/LM Studio runs reach the top of the band. Jeff Geerling's beowulf-ai-cluster work documents the perf-tuning that gets there.
• The Phi-3.5 "up to 61 tok/s" figure is AMD Developer's own vendor self-reported number (from its Ryzen AI Max+ cluster write-ups) — we label it as such rather than treating it as independent.

For the deep dive on the 120B MoE case specifically — quant choices, which box, real setup — see how to run GPT-OSS 120B locally on a mini PC.

Prefill vs decode: the number benchmarks hide

Every tok/s figure above describes decode — the token-by-token generation speed after your prompt is processed. There's a second number that matters and that most benchmarks quietly omit: prefill, the time to process the input prompt before the first token appears.

Prefill is compute-bound, not bandwidth-bound, and on unified-memory boxes like Strix Halo it is generally slower than on a discrete GPU. For short chat prompts you'll never notice. But feed it a long context — a big RAG retrieval, a whole code file, a lengthy system prompt — and prefill adds real latency before generation starts. A box that decodes at 80 tok/s can still make you wait a beat on first token if the prompt is thousands of tokens long.

We won't fabricate a prefill tok/s figure — it varies too much with context length, batch, and runtime — but the honest expectation is: prefill on Strix Halo is generally slower than a dGPU, and it's the caveat that matters most for RAG and coding workloads. If your use is interactive chat with short prompts, ignore it. If it's long-context retrieval, factor in first-token latency on top of the decode numbers above.

Which box changes the number — and which doesn't

Here's a mistake that costs money: agonizing over which Strix Halo box is "fastest." For peak tokens/sec, they're effectively identical, because they run the same Ryzen AI Max+ 395 silicon and the same ~256 GB/s memory:

• GMKtec EVO-X2 — the flagship, quiet dual-M.2 appliance ($1,999–$2,199)
• Framework Desktop — cheapest credible 128GB box, standard mini-ITX ($1,999)
• Beelink GTR9 Pro — dual 10GbE + vapor-chamber cooling ($1,899–$1,999)
• Minisforum MS-S1 Max — best I/O + 320W PSU ($2,879–$3,039)
• HP Z2 Mini G1a — business-grade PRO silicon, ECC, 3-yr warranty ($3,300–$3,734)

What actually differs between them is sustained throughput, not peak. Cooling and power delivery decide how long a box holds its peak tok/s before thermal throttling drags it down. The Beelink GTR9 Pro's vapor-chamber dual-fan cooler and the Minisforum MS-S1 Max's 320W PSU and oversized heatsink are built to sustain ~120W inference longer than a palm-sized box can. Notebookcheck praised the GTR9 Pro's "silent 120B LLM performance at 120W," and ServeTheHome measured it sustaining ~120W inference at roughly 36–41 dBA (while also flagging a 10GbE NIC crash under sustained iGPU load — a driver issue worth knowing).

The bottom line: peak tok/s is a silicon fact, identical across boxes; sustained tok/s is a cooling fact, where the better-built boxes win. If you'll run long, back-to-back inference sessions, the cooling matters. If you hit the box in bursts, any of them delivers the same peak.

Two adjacent platforms shift the number in opposite directions. The NVIDIA DGX Spark (273 GB/s, CUDA) is a hair faster on bandwidth and wins on software/clustering, not raw single-box speed — we break that down in DGX Spark vs Strix Halo. And if you want genuinely higher token speed at 128GB, the Mac Studio M4 Max (up to 546 GB/s) is the move.

Budget-tier reality: the 7–13B boxes

Not everyone needs 128GB. If you'll never touch a 30B+ model, the sub-$600 boxes generate 30–45 tok/s on 7–13B dense models — snappy and interactive — for a quarter of the money. The Beelink SER8 ($449–$599, Ryzen 7 8845HS, 32GB) and GMKtec M6 Ultra ($429–$549, Zen 4, 32GB) are the value picks here. Their ceiling is real — 32GB of RAM and modest bandwidth lock them out of 30B+ models — but for small local models and always-on agents they're the right buy, and you're not paying for capacity you won't use. For the full tiered breakdown, see the best mini PC for local LLMs in 2026.

So is Strix Halo fast enough for you?

Now the decision framework, using the numbers above rather than a vibe:

The reframe that should change your purchase: the scary "5 tok/s" number only applies to dense 70B models — which, in 2026, fewer and fewer people actually run. The models that matter now are MoE, and on MoE the same $2,000 box is comfortably interactive. Buy Strix Halo for the workload you'll actually have, not the dense-70B strawman the spec sheet implies. If you run MoE — and you probably will — it's fast enough, and then some.

Still deciding which specific box, or whether a unified-memory machine beats a GPU build for your workload? Start with our GMKtec EVO-X2 review for the flagship pick, or the tiered buyer's guide for every budget.

Bottom line

"How fast is Strix Halo?" is the wrong question — the right one is "how fast is Strix Halo on the model I run?" On the Ryzen AI Max+ 395, a dense 70B crawls at ~5 tok/s while a 30B MoE model like Qwen3-30B-A3B flies at 70–100 tok/s and GPT-OSS 120B holds ~31 tok/s — a 15× spread on identical silicon, all explained by memory bandwidth (~215 GB/s real) and how many weights each architecture reads per token. The box you pick barely moves peak speed; the model architecture moves it enormously. For 2026's MoE-first reality, Strix Halo is fast enough and the value pick at ~$2,000. Just buy it knowing what it's actually good at.