GGUF

GGUF is the model file format used by llama.cpp and the tools built on it (Ollama, LM Studio), designed to package a quantized model plus its metadata in a single file that runs on CPU, GPU, or unified memory. It replaced the older GGML format and is the de facto standard for local inference: when you pull a model in Ollama or download one from Hugging Face for LM Studio, you're almost always getting a GGUF. Files are named by quantization level — e.g. Q4_K_M — so you can pick the size/quality trade-off that fits your box. Apple's MLX ecosystem uses its own formats instead.

GGUF is what you'll actually download and run on an AMD or NVIDIA box. Match the GGUF quant to your allocatable memory — grab the Q4_K_M build of a 70B if you've got ~96GB to spare, a smaller quant if not.