GGUF
GGUF: a single-file format that holds an AI model and everything needed to load it, designed for running models locally.

GGUF is a file format designed to distribute and run artificial intelligence models locally. A single file contains the model and everything needed to load it. It's the de facto format for language models you run yourself, outside online services.
Technical definition
One file, all included
Its specification sets three goals: single-file deployment, with no dependency on side files; an extensible format, to which information can be added without breaking existing models; and fast loading. GGUF succeeded several earlier formats from the same project, including GGML.
Readable metadata
Besides the model's numbers, the file carries structured metadata as key-value pairs: the architecture, the maximum context size, the chat template, the quantization level. A tool can therefore load a model without being handed its configuration separately.
Quantization's companion
The same model is usually offered as several GGUF files, one per quantization level. You pick the one that fits in your machine's memory. The file name indicates that level with a suffix.
Current challenges
Running a model at home
GGUF has made open models easier to access: where you used to have to assemble a technical environment, a file and a tool are now enough. For an organization, this opens up uses where data never leaves the workstation or the internal server.
What the file decides for you
The embedded metadata drives the model's behavior. An incomplete or altered chat template changes how the model handles instructions or tool calls. When a local model behaves oddly, the file is as serious a suspect as the model.
A supply chain to watch
Most GGUF files in circulation don't come from the model publishers. They are produced and republished by third parties, sometimes with modifications. Checking a file's provenance before deploying it is the same hygiene as for any software dependency.
Frequently Asked Questions
What is a GGUF file used for?
To run a language model on your own machine, with tools such as llama.cpp, Ollama, LM Studio or GPT4All. The file contains the model and its settings: you download it, load it, and it works. Models in this format are most often quantized, meaning slimmed down to fit on an ordinary graphics card.
What is the difference between GGUF and safetensors?
Both store a model's numbers. The Hugging Face documentation sums up the gap: safetensors contains only the tensors, whereas GGUF adds a standardized set of metadata. Safetensors is the format of original models, used for training. GGUF is an inference format, designed for fast loading.
Where can you find models in GGUF format?
Mainly on Hugging Face, which lets you filter repositories by this format and inspect a file's contents before downloading it. Many of these files are published by independent contributors rather than by the model's publisher: check the author, the original model and the quantization level.