Reference
AI Hardware Glossary
Every term you need to understand before buying AI hardware.
A
Apple Silicon (M3 Ultra / M4)
Apple's M-series chips are the bandwidth leaders among unified-memory machines. The M3 Ultra runs at 819 GB/s — roughly 3× any Strix Halo or GB10 box — and at launch scaled to 512GB, enough to hold DeepSeek R1 671B at 4-bit in memory (~17–18 tok/s, under 200W). The M4 family spans a wide range: 120 GB/s on the base Mac mini (a 24GB, ~7–14B-class entry point), 273 GB/s on the M4 Pro, and 410–546 GB/s with up to 128GB on the M4 Max. The catch is software — no CUDA, so you run MLX, llama.cpp, or Ollama.
C
Context Window
The context window is how many tokens a model can consider at once — its working memory for the prompt plus everything it has generated so far. Larger contexts let you feed whole documents or long chat histories, but they cost memory: the key-value cache grows with context length and sits in the same unified-memory pool as the weights. On a 96GB-allocatable box, a long context eats into the budget you'd otherwise spend on a bigger model. Long prompts also add prefill time before the first token, which is slower on unified-memory boxes than on discrete GPUs.
CUDA
CUDA is NVIDIA's GPU-compute platform and the de facto standard most AI software targets first. Its ubiquity is the recurring theme in this catalog's trade-offs: the NVIDIA GB10 boxes (DGX Spark, ASUS Ascent GX10) are 'CUDA-native,' while every AMD Strix Halo box and every Apple Mac is flagged 'no CUDA' — meaning some tools assume NVIDIA and need a workaround elsewhere. On AMD you fall back to ROCm or Vulkan; on Apple you use MLX or Metal. For mainstream models via Ollama or LM Studio this rarely bites, but niche or cutting-edge tooling often ships CUDA-first.
G
GPU-Allocatable VRAM
On a unified-memory box, GPU-allocatable VRAM is the slice of the shared RAM pool the graphics processor is allowed to treat as video memory. AMD's Ryzen AI Max+ 395 boxes ship 128GB of LPDDR5X and let you assign up to 96GB of it to the GPU — the headroom that loads 70B-class and larger models. The rest stays reserved for the CPU and OS. It's a firmware/driver setting, not a fixed hardware limit, which is why the same 128GB box can be configured for more or less GPU memory.
GGUF
GGUF is the model file format used by llama.cpp and the tools built on it (Ollama, LM Studio), designed to package a quantized model plus its metadata in a single file that runs on CPU, GPU, or unified memory. It replaced the older GGML format and is the de facto standard for local inference: when you pull a model in Ollama or download one from Hugging Face for LM Studio, you're almost always getting a GGUF. Files are named by quantization level — e.g. Q4_K_M — so you can pick the size/quality trade-off that fits your box. Apple's MLX ecosystem uses its own formats instead.
GB10 (Grace Blackwell)
GB10 is NVIDIA's Grace Blackwell superchip for desktop AI boxes — the CUDA-native alternative to Strix Halo. It combines a 20-core Arm CPU with a Blackwell GPU and 128GB of coherent unified memory, and ships in the NVIDIA DGX Spark and ASUS Ascent GX10. Its headline '1 PFLOP FP4' is a sparse figure; dense compute is roughly half (~500 TFLOPS, about RTX 5070-class), and at 273 GB/s memory bandwidth is the real ceiling. A 200GbE ConnectX-7 NIC lets two units cluster to run 405B-class models locally.
I
Inference vs Training
Inference is running a trained model to generate output; training is building the model's weights in the first place. These mini PCs are inference machines — sized to load and run existing models, not to train new ones from scratch, which needs far more compute and high-bandwidth memory than any unified box provides. Even the 128GB boxes and the M3 Ultra are described for running models (a 70B at Q4, or DeepSeek R1 671B on a 512GB Ultra), not pretraining them. Light fine-tuning of small models is feasible, especially via MLX on Apple Silicon, but serious training belongs on datacenter GPUs.
L
LPDDR5X
LPDDR5X is the low-power mobile DRAM soldered into these unified-memory boxes, and its speed grade sets their bandwidth ceiling. The Ryzen AI Max+ 395 boxes use LPDDR5X-8000 (the HP Z2 Mini runs 8533) for 256 GB/s theoretical / ~215 GB/s real across a 256-bit bus — about 2× a normal desktop APU. The GB10 boxes and Apple's chips use LPDDR5X too, reaching 273 GB/s on the DGX Spark and up to 819 GB/s on the M3 Ultra via much wider buses. Because it's soldered, you choose the capacity once at purchase — there's no upgrade path.
LM Studio
LM Studio is a desktop app for running local LLMs with a graphical interface — a model browser, a chat window, and a one-click local server, all without the command line. It downloads GGUF models from Hugging Face, lets you pick the quantization visually, and shows whether a model will fit your memory before you load it. It runs on Windows, macOS, and Linux, making it a natural fit for the Strix Halo and Apple boxes alike. Under the hood it uses llama.cpp, and MLX builds on Apple Silicon.
llama.cpp
llama.cpp is the open-source C/C++ inference engine that most local-LLM tools are built on — Ollama and LM Studio both wrap it. It introduced the GGUF format and the widely-used Q4_K_M-style quantization schemes, and it runs on almost anything: CPU, AMD (via ROCm or Vulkan), NVIDIA (CUDA), and Apple (Metal). That broad hardware support is why the same quantized model runs across every box in this catalog. Serious users run it directly for maximum control over threading, context, and GPU offload.
M
Memory Bandwidth
Memory bandwidth is how fast the processor can read the model's weights out of memory, measured in GB/s, and for local LLM inference it's the single number that best predicts tokens/sec. Every weight is read once per token generated, so bandwidth divided by model size roughly caps your speed. That's why a Strix Halo box at ~215 GB/s real produces single-digit tokens/sec on a dense 70B, while an M3 Ultra at 819 GB/s is several times faster on the same model. Discrete GPUs hit 800–1000 GB/s; the NVIDIA GB10 boxes sit at 273 GB/s despite being CUDA-native.
MLX
MLX is Apple's open-source array framework for machine learning, built to exploit Apple Silicon's unified memory and Metal GPU. It's the fastest path to running and fine-tuning LLMs on a Mac, and it's why an M3 Ultra or M4 Max can deliver its full 546–819 GB/s of bandwidth to a model rather than leaving performance on the table. Tools like LM Studio use MLX builds on Apple hardware alongside GGUF. It only runs on Apple Silicon — there's no MLX on AMD or NVIDIA boxes.
MoE (Mixture of Experts)
A Mixture of Experts (MoE) model routes each token through only a fraction of its total parameters — a set of specialized 'expert' sub-networks — so a huge model runs at the speed of a much smaller one. This is why a 512GB M3 Ultra can serve the 671-billion-parameter DeepSeek R1 at a usable ~17–18 tok/s: only a slice of those parameters activates per token, so the bandwidth demand is far below the full model size. MoE models still need enough memory to hold all their weights, which is where the 128GB and larger unified boxes earn their keep. Dense models of the same parameter count would run much slower.
N
NPU / TOPS
An NPU (Neural Processing Unit) is a dedicated AI accelerator alongside the CPU and GPU, and TOPS (trillion operations per second) is the unit used to rate it. The Ryzen AI Max+ 395 boxes carry a 50-TOPS XDNA 2 NPU; NVIDIA's GB10 and Apple's Neural Engine take different approaches to the same idea. TOPS measures low-precision throughput for on-device AI tasks — background features, vision, small models — not the memory-bound work of large LLM generation, which still leans on GPU bandwidth. So a high TOPS number doesn't translate into faster 70B tokens/sec.
O
Ollama
Ollama is the most popular way to run local LLMs — a command-line tool and background server that pulls quantized GGUF models with one command (ollama run llama3) and exposes an OpenAI-compatible API on localhost. It handles model downloading, quantization selection, and memory management, which is why it's the default recommendation for these mini PCs across AMD, NVIDIA, and Apple hardware. On unified-memory boxes it loads the model into the shared pool and runs on the GPU where supported. It's the easiest on-ramp; LM Studio adds a GUI, and llama.cpp sits underneath both.
P
Prompt Processing (Prefill)
Prompt processing, or prefill, is the work a model does reading your input before it generates the first token — as opposed to the token-by-token decode that follows. It's more compute-bound than the bandwidth-bound decode phase, and it's a known weak spot for unified-memory boxes: the Mac Studio M3 Ultra, for instance, has slow prefill on long contexts versus GPU rigs, despite its 819 GB/s bandwidth. The longer your prompt, the longer you wait for that first token. Discrete GPUs, with far more compute, feel snappier here even when generation speed is similar.
Q
Quantization
Quantization shrinks a model by storing its weights at lower numerical precision — commonly 4-bit (Q4) instead of the original 16-bit — cutting the memory footprint by roughly 4× with a small, usually acceptable quality loss. It's the technique that makes big models fit these boxes: at ~2 bytes per weight a 70B model is ~140GB in 16-bit, but ~4× smaller (~40GB) at 4-bit, small enough for a 96GB-allocatable Strix Halo box. At the extreme, a 512GB M3 Ultra runs the 671-billion-parameter DeepSeek R1 at 4-bit entirely in memory. Lower bit-widths (Q3, Q2) shrink further but degrade quality faster.
R
Ryzen AI 9 (Strix Point)
Ryzen AI 9 is AMD's Strix Point tier — the thin-and-light laptop and handheld sibling of the Ryzen AI Max+ 395 'Strix Halo' in these mini PCs. It shares the same XDNA 2 NPU generation (the 50-TOPS class that qualifies for Copilot+ AI features) but uses a narrower dual-channel memory bus rather than Strix Halo's 256-bit controller, so its memory bandwidth is roughly half — a real cap for local LLM inference. It also has no large GPU-allocatable memory pool, so it's built for lightweight on-device AI, not for loading 70B-class models. If a listing says 'Ryzen AI 9' rather than 'Ryzen AI Max+ 395,' it is not one of the 128GB big-model boxes.
ROCm
ROCm is AMD's GPU-compute stack — its answer to NVIDIA's CUDA — and it's the software layer that decides how well the Radeon 8060S iGPU in a Strix Halo box accelerates AI work. On the Ryzen AI Max+ 395 boxes, ROCm/Linux GPU-compute is still rough compared with CUDA's maturity, which is the recurring caveat on AMD hardware: the memory capacity is there, but the software path is less polished. Many users route around it via llama.cpp's Vulkan backend instead. It's improving quickly but isn't yet CUDA-smooth.
S
Strix Halo (Ryzen AI Max+ 395)
Strix Halo is AMD's codename for the Ryzen AI Max+ 395, the APU behind nearly every 128GB local-AI mini PC. It pairs a 16-core Zen 5 CPU, a Radeon 8060S iGPU (40 CU, RDNA 3.5), and a 50-TOPS XDNA 2 NPU with a 256-bit LPDDR5X-8000 memory controller — 256 GB/s theoretical, ~215 GB/s real. That memory setup, roughly 2× a normal desktop APU, is what lets boxes like the GMKtec EVO-X2, Framework Desktop, and HP Z2 Mini expose up to 96GB as VRAM and run 70B-class models. Its weakness is bandwidth, far below a discrete GPU, so dense 70B inference runs in the single-digit tokens/sec.
Soldered Memory
In every unified-memory box here, the RAM is soldered to the board (LPDDR5X) and cannot be upgraded after purchase. That's a physical consequence of the wide, fast memory buses these chips need — a Strix Halo box's 256-bit LPDDR5X-8000 or an M3 Ultra's 819 GB/s setup only works because the memory sits right next to the processor. So you must buy the 128GB (or whatever) config up front; there's no adding more later. It's the single most important 'buy it right the first time' constraint on these machines.
Sparse vs Dense (FP4)
Sparse and dense describe whether a headline compute figure counts skipped-zero operations or every operation. It matters because NVIDIA's GB10 '1 PFLOP FP4' rating is a sparse number — dense compute is roughly half, ~500 TFLOPS, about RTX 5070-class. FP4 is a 4-bit floating-point format used for that peak rating. When you compare AI accelerators, a sparse figure and a dense one aren't the same measurement, and the sparse one is always the bigger, more marketable number.
T
Tokens per Second
Tokens per second (tok/s) is the throughput of text generation — how many word-pieces the model emits each second — and it's the number that decides whether local inference feels usable. As a rough rule of thumb one token is about three-quarters of a word, so 10 tok/s reads faster than most people do and single-digit tok/s feels sluggish for chat. On these boxes it's set mostly by memory bandwidth ÷ model size: a dense 70B on a ~215 GB/s Strix Halo box lands in the single digits, while a 512GB M3 Ultra runs DeepSeek R1 671B (a sparse MoE) at ~17–18 tok/s under 200W. Long prompts add prefill latency before the first token appears.
U
Unified Memory
Unified memory is a single pool of RAM shared by the CPU, GPU, and NPU, so the graphics processor reads model weights directly instead of copying them across a PCIe bus into separate VRAM. It's what lets a box like the GMKtec EVO-X2 expose 128GB of LPDDR5X to the GPU — up to 96GB assignable as VRAM — and load 70B-class models no 24–32GB discrete card can hold. Apple pioneered the approach; the M3 Ultra scales the same idea to 819 GB/s. The trade-off is bandwidth: unified LPDDR5X runs ~215–273 GB/s on the AMD and NVIDIA boxes versus 800–1000 GB/s on a discrete GPU, so you gain capacity but not raw speed.
Ready to Buy?
Now that you know the terms, find the right hardware for your AI workload.