GGUF vs Safetensors: What Changes on Disk and in Memory

GGUF and Safetensors are not the same kind of file. This page compares disk size, load-time VRAM behavior, quantizability, and which toolchains each suits.

By VRAM Fit editorial Published 2026-09-17 Updated 2026-09-17

GGUF and Safetensors are often compared as if one is newer or better than the other. They are different kinds of artifact. Safetensors is a container for raw weights, usually at full precision, that an engine loads and then may quantize. GGUF is a single file that is already quantized and packaged for a specific family of loaders. The difference shows up on disk, in memory at load time, and in which toolchains expect which format. This page compares them along those axes and clears up the common belief that GGUF simply means “smaller”.

What each format is

Safetensors is a safe, simple tensor container. It stores the model’s weights, typically in their native training precision such as FP16 or BF16, with a header describing the tensors. It is a raw weight container: fast to load, free of arbitrary code, but not itself a quantization. Before efficient local inference, the weights usually need conversion or quantization into another format.

GGUF is a format built for llama.cpp and the tools built on it. A distributed GGUF file is normally already quantized — the bits-per-parameter decision is baked in — and packaged with the metadata the loader needs. That is why a GGUF can be loaded directly by llama.cpp or Ollama without a separate quantization step.

The key distinction: Safetensors holds weights; GGUF holds quantized, loader-ready weights. One is a container, the other is a prepared artifact.

Format comparison

The table is qualitative on purpose. Exact disk sizes vary by model and by which quantization a given GGUF carries, and the model file size lookup gives representative numbers per model and tier.

Format	Typical use	Disk volume	VRAM on load	Directly quantizable?	Suited toolchain
Safetensors	Raw, often full-precision weights	Large (full precision)	Weights plus cache plus overhead, after conversion	Container only; quantize separately	PyTorch, Transformers, many training and serving stacks
GGUF	Pre-quantized, loader-ready single file	Varies by tier (smaller when quantized)	Quantized weights plus KV cache plus overhead	Already quantized in the file	llama.cpp, Ollama, and compatible local loaders

Neither format changes the standing rule that loaded VRAM equals weights plus KV cache plus runtime overhead. The format decides how the weights arrive; the memory math afterward is the same.

GGUF is not “smaller”, it is “already quantized”

The most common misunderstanding is that GGUF is a compression format that makes models small. It is not. A GGUF is smaller than a full-precision Safetensors because the GGUF carries quantized weights, while the Safetensors commonly carries full-precision weights. Put a full-precision model into GGUF and it would be as large as the full-precision Safetensors.

So the size difference comes from quantization, not from the container. The weight formula still applies: weight VRAM is the parameter count in billions times the bits per parameter divided by eight. A Q4_K_M GGUF uses about 4.5 bits per parameter, which is why it is smaller than an FP16 Safetensors at 16 bits per parameter. The quantization comparison maps tiers to bits per parameter, and the quantization quality loss page covers what the lower bits cost in output.

Why one model ships as many GGUF files

A single base model is released as several GGUF files, each at a different quantization tier. Q2_K, Q3, Q4_K_M, Q5, Q6, Q8 and others sit at different points on the quality-versus-size curve. A user with a small card picks a lower tier that fits; a user with a large card picks a higher tier for more quality.

These tiers are not different models. They are the same weights at different bits per parameter. The file on disk tells you the tier, and therefore the approximate weight VRAM, before you load. Real GGUF files often run a few percent larger than the simple formula predicts because token embeddings and the output tensor are kept at higher precision; the rule “use the actual file size as the source of truth” applies.

Download size is not load size

A second misunderstanding is equating the file’s download size with the VRAM the model needs. They are different numbers. The file holds the weights. After loading, the engine also allocates the KV cache for the chosen context and reserves runtime overhead. So:

File on disk = quantized weights only.
Loaded VRAM = weights + KV cache + engine headroom.

The KV cache is a separate, often large, line item that grows with context length and is typically held at 16-bit regardless of the weight quantization. The KV cache size page shows how quickly it grows, and the context length cost page ties it to session length. The runtime overhead is small but real and should not be forgotten when a card is near full.

Which to choose

The choice follows the runtime and hardware, not a ranking of formats. A setup built on llama.cpp or Ollama uses GGUF directly, because that is what those loaders expect and the files are already quantized for them. A pipeline built on a framework that trains or serves from Safetensors uses that format, often converting to GGUF only when moving to local inference.

There is no “better” format in the abstract. The question is which toolchain is in front of you and what your memory budget is. If the goal is to run locally on a constrained card, a GGUF at a fitting tier is the direct path. If the goal is training, fine-tuning, or a server stack that expects raw weights, Safetensors is the natural container and quantization happens as a separate step. The run locally guide lays out the path from zero to a running model across these choices.

Memory behavior at load

Both formats end at the same memory math once loaded. The difference is only what the weights weigh when they arrive. A GGUF arrives pre-quantized, so its weight line is already at the chosen bits per parameter. A Safetensors that is converted and quantized on the way in arrives at the same weight line after that step. Either way, the KV cache and overhead are added on top, and the VRAM calculator estimates the total from the weight size and context.

For very large models, the file format also interacts with how the model is split. Multi-GPU setups and offload change where the weights live, as the multi-GPU VRAM page and the CPU offload performance page describe, but those decisions are independent of whether the source was GGUF or Safetensors.

Practical notes

Check the actual file size, not the tier name, when estimating, because real files carry small overhead beyond the simple formula.
Match the quantization tier to the card’s usable memory, remembering that usable VRAM is below the nominal figure; the GPU VRAM reference lists nominal values and the caveat that nominal is not available.
If a load fails with out-of-memory, the cause is usually the combined weights plus KV cache plus overhead, not the file format. The Ollama out of memory guide walks through diagnosing that.

What this page does not tell you

This page does not give exact disk or VRAM numbers for any specific model, because those depend on the parameter count, the quantization tier, and the small overhead real files carry; the model file size lookup holds representative figures. It does not rank the formats as better or worse, and it does not recommend a quantization tier or a graphics card. Conversion tooling specifics, training-from-Safetensors workflows, and distributed or multi-GPU loading are outside this page’s scope. The claim that GGUF equals “already quantized” describes the files distributed for local use, not a property enforced by the format specification on every possible GGUF.

Asked before trying it

Is a GGUF file smaller than a Safetensors file of the same model?

Not by format. GGUF is usually smaller because it is already quantized, while a Safetensors file commonly ships the full-precision weights. The format does not shrink anything; the quantization inside the GGUF does. A full-precision GGUF would be as large as a full-precision Safetensors.

Can Safetensors be loaded directly by llama.cpp and Ollama?

Not in the same direct way. Safetensors is a raw weight container, typically high precision, and usually needs conversion or quantization before efficient local inference. GGUF is a single file already prepared for llama.cpp-family loaders.

Why are there several GGUF files for one model?

Each file is a different quantization tier. Q2_K, Q3, Q4_K_M, Q5, Q6, Q8 and others trade quality for size, so one base model ships as several GGUF files at different points on that curve. The [model file size lookup](/model-file-size-lookup/) lists representative tiers.

Is the download size the same as the VRAM used after loading?

No. The loaded model needs VRAM for the weights plus the KV cache for the context plus engine overhead. The file on disk is only the weights; the cache and runtime add on top, so loaded VRAM is a different and larger number.

Which format should I choose?

It depends on the runtime and hardware, not on which format is intrinsically better. A llama.cpp or Ollama setup uses GGUF directly. A pipeline built on a framework that expects Safetensors uses that. The choice follows the toolchain, not a quality ranking.

Does GGUF mean the model is already quantized?

In practice, yes, the GGUF files distributed for local use are pre-quantized. That is what makes them directly loadable at a known size. The quantization is baked in, which is different from a raw weight container that you quantize yourself.