Ollama reports VRAM usage that depends on three things added together: the model weights, a KV cache sized to the active context window, and a small runtime headroom. A single model does not have one VRAM number. This page estimates that total for common models and quantization tiers, using a fixed 4K context, with the same formula the rest of the site uses. Every figure below is a calculation from published model shapes and quant bit rates, not a reading from a live card. The number that matters for a specific machine is whatever ollama ps reports after the model loads.
How the estimate is built
Three terms make up the total. The weights scale with parameter count and quant bits. The KV cache scales with the model’s attention shape and the chosen context length. The headroom is a fixed add-on for the runner and driver allocations.
weight VRAM (GB) = params(B) × bits-per-param ÷ 8
KV cache (GB) = 2 × layers × KV heads × head dim × context × (KV bits ÷ 8)
total (GB) = weights + KV cache + headroomThe weight term uses the quant tier’s bits-per-parameter. The KV term uses the model’s attention shape taken from its public configuration and assumes a 16-bit cache, which is the common default across engines. A fixed headroom of 0.5 GB is added on top to cover the Ollama runner and the driver and kernel allocations that sit beside the model. Every total in the table is the weight size for that tier (decimal GB) plus the model’s 4K KV figure plus 0.5 GB.
Two caveats apply to the whole table. First, real GGUF files are often 5 to 10 percent larger than the weight figure here, because token embeddings and the output tensor are typically kept at higher precision than the bulk of the weights. The file on disk is the real lower bound, and the model file size lookup shows those on-disk sizes directly. Second, these are formula estimates. The actual ollama ps reading depends on the exact GGUF, the Ollama build, and how many layers end up on the GPU.
The model × quant table (4K context, estimate)
The table below pairs each model with several quant tiers. Each cell is an estimate: the weight size for that tier plus the model’s 4K KV cache plus 0.5 GB of headroom. The KV cache does not change with the quant tier. The model shapes are taken from each model’s public config; to verify exact layer and head counts, check that model’s own config.json.
| Model | Params | 4K KV | Q3_K_M | Q4_K_M | Q5_K_M | Q8_0 | F16 |
|---|---|---|---|---|---|---|---|
| Llama 3.2 3B | 3B | 0.44 GB | 2.2 GB | 2.6 GB | 3.0 GB | 4.1 GB | 6.9 GB |
| Mistral 7B | 7B | 0.5 GB | 4.1 GB | 4.9 GB | 5.8 GB | 8.4 GB | 15.0 GB |
| Llama 3.1 8B | 8B | 0.5 GB | 4.5 GB | 5.5 GB | 6.5 GB | 9.5 GB | 17.0 GB |
| Qwen 2.5 14B | 14B | 0.75 GB | 7.4 GB | 9.2 GB | 10.9 GB | 16.2 GB | 29.3 GB |
| Qwen 2.5 32B | 32B | 1.0 GB | 15.5 GB | 19.5 GB | 23.5 GB | 35.5 GB | 65.5 GB |
| Llama 3.3 70B | 70B | 1.25 GB | 32.4 GB | 41.2 GB | 49.9 GB | 76.2 GB | 141.8 GB |
These are estimates calculated from the formula, not measured values. The actual occupancy should be confirmed with ollama ps on the target machine. Note that the KV column is identical inside a row across quant tiers: quantizing the weights does not shrink the cache.
Why weights and KV are separate budgets
The single most useful idea on this page is that the weight budget and the KV budget move independently. The weights are fixed once a tag is chosen. The KV cache grows with the context window and, when serving, with the number of concurrent sessions. A model that fits comfortably at 4K can become impossible at 128K purely from the cache, with the weight number unchanged. This separation is why a coarse “weight × fixed multiplier” calculator undershoots at long context and why this site keeps the two terms distinct. The KV cache size explained page works through the cache term on its own.
How Ollama decides how many layers go on the GPU
Ollama runs llama.cpp underneath, and both split a model between the GPU and system RAM using a layer-offload count. In Ollama this is exposed as num_gpu. The engine tries to keep as many layers as fit on the GPU; any layer that does not fit is computed on the CPU and stored in system memory. The weights still have to live somewhere, either in VRAM or in RAM, so the total weight memory does not disappear when layers are offloaded — it moves.
The practical reading of the table above is the pure-GPU case, where every layer fits on the card. Once offload begins, the GPU-side number drops and the system-RAM number rises by the same weight amount. Whether a model “fits” on a card is therefore partly a speed question, not only a capacity question. A 70B model at Q4_K_M needs about 41 GB of weight space; on a 24 GB card that space is met by combining VRAM and RAM through offload, not by fitting it all on the GPU. The offload layers guide covers how the split trades memory against speed in detail, and the multi-GPU page describes how two cards split the same weight space.
num_gpu is most usefully read as a budget dial. Raising it keeps more compute on the GPU and runs faster; lowering it trades speed for fit. When a model is near the limit, lowering num_gpu is the move that keeps it running without changing the weights or the context. This is the same mechanism as llama.cpp’s -ngl flag, so tuning one teaches the other.
Why the default context is smaller than the model max
A model card may advertise a 128K token context window. Ollama’s default context is far smaller, often 2048 tokens, and only sometimes raised in newer builds — but it stays well below the model’s advertised maximum. Because the KV cache scales linearly with context length, the memory Ollama reports at default context is a small fraction of what the same model would consume at its maximum context.
This is why ollama ps on a freshly pulled model often shows a number much lower than the model’s theoretical peak. The weights are fixed, but the cache that sits beside them is tiny at a 2K or 4K window. Reading the live number tells you the cost of the current context, not the cost of the model’s full capability. To see the difference, set a larger num_ctx and watch the reading climb: the weights stay put while the cache grows. At 128K the cache alone for an 8B-class model is about 16 GB, larger than the weights of many quant tiers. The context length cost page works through that climb for several lengths and shows how batching multiplies it further.
The default-small-context behavior also explains a common surprise: a model “loads fine” but then throws out of memory only on a long reply. At load time the small default context reserves a small cache; the long reply pushes the context out, the cache grows, and the reserved budget is exceeded mid-generation. The fix is to set a context the card can actually hold, not to reload the same model expecting a different result.
How much the quant variant of one tag changes memory
The same base model ships under several tags that differ only in quantization. For an 8B model the weights alone range from about 3.5 GB at Q3_K_M to about 8.5 GB at Q8_0, with Q4_K_M near 4.5 GB — roughly a 2.4× spread between the lowest and highest common tiers, before any KV or headroom. At 4K context, including the fixed cache and headroom, the total for that one model spans roughly 4.5 GB at Q3_K_M to 9.5 GB at Q8_0.
The takeaway is that the quant tier is the largest single lever on a single-model load, because it scales the weights directly while the KV cache at a fixed context stays constant. Two cards that both “run an 8B model” can be discussing very different memory footprints depending on which tag was pulled. For a 70B model the same logic produces a far wider absolute gap: Q3_K_M lands near 32 GB of weights while Q8_0 approaches 74 GB, a difference larger than an entire 24 GB or 32 GB card. The quantization comparison explains how each tier trades precision for size, and the quantization quality loss page describes what that trade costs in output behavior, with source grades kept honest.
Why a tag is not a single number
Even within one quant family, the exact GGUF matters. K-quant variants mix precisions across tensor groups, and the embedding and output tensors are usually left higher. A Q4_K_M from one source can differ from another by several percent in file size, and Ollama may pull a specific build for a given tag over time. That is why the table here rounds to one decimal and why the file on disk remains the authoritative weight figure. The GGUF versus safetensors page covers how the two formats differ in what they store, which also affects the effective size.
Reading the real number
After a model loads, ollama ps reports the resident set for that model on the machine. That reading is the only one tied to the actual hardware, build, and tag in use. The estimates on this page are a planning aid: they say what a model is likely to need, they do not replace the live reading. When the two disagree, trust ollama ps and the specific GGUF file size over any rounded table value. The measure VRAM usage page lists how to read memory on each platform, and the GPU VRAM reference lists nominal card sizes so the live reading can be compared against the card’s stated capacity.
What this page does not tell you
- It does not contain measured readings from any specific card. Every total here is a formula estimate from published shapes and bit rates.
- It does not cover differences between Ollama versions or between Ollama and other runtimes, whose memory handling changes release to release.
- It does not describe the speed effect of CPU offload. Moving layers off the GPU frees VRAM but changes generation speed, and that trade is outside this page’s scope.