llama.cpp

llama.cpp is the open-source C/C++ inference engine that most local-LLM tools are built on — Ollama and LM Studio both wrap it. It introduced the GGUF format and the widely-used Q4_K_M-style quantization schemes, and it runs on almost anything: CPU, AMD (via ROCm or Vulkan), NVIDIA (CUDA), and Apple (Metal). That broad hardware support is why the same quantized model runs across every box in this catalog. Serious users run it directly for maximum control over threading, context, and GPU offload.

You may never call llama.cpp directly, but it's the engine deciding how fast and how compatibly your model runs. When a box's 'AMD support' is described as rough, it's usually the llama.cpp/ROCm path being referred to.