When a model does not fit, the size of its weights is the first number to pin down. This page is a static lookup: common model scales across the usual GGUF quantization tiers, computed from bits per parameter, with a reverse table that starts from your VRAM budget instead of from the model. The sizes are estimates from a formula; the real file is the authority.
The formula, stated once
All sizes on this page use one convention, stated here so it can be checked by hand:
weight size (GB) = parameters (billions) × bits per parameter ÷ 8where 1 GB = 1024 MB = 1073741824 bytes (2 to the 30), following the site-wide unit rule. The bits per parameter are the common GGUF tier approximations, not exact per-model figures. Multiply, divide by eight, and you have gigabytes of weights.
A quick sanity check: Q4_K_M stores about 4.5 bits per weight, so a 7B model is 7 × 4.5 ÷ 8 ≈ 3.94 GB, commonly rounded to about 3.9 GB. The same tier on 70B is 70 × 4.5 ÷ 8 ≈ 39.4 GB. The math scales linearly with parameter count.
Common model sizes by quantization
The table is the lookup this page exists to provide. Rows are model scales; columns are tiers. Every cell is an estimate from the formula above and the approximate bits listed in the header row.
| Model | Q2_K (2.6) | Q3_K_M (3.5) | Q4_K_M (4.5) | Q5_K_M (5.5) | Q6_K (6.6) | Q8_0 (8.5) | F16 (16) |
|---|---|---|---|---|---|---|---|
| 7B | 2.3 GB | 3.1 GB | 3.9 GB | 4.8 GB | 5.8 GB | 7.4 GB | 14.0 GB |
| 13B | 4.2 GB | 5.7 GB | 7.3 GB | 8.9 GB | 10.7 GB | 13.8 GB | 26.0 GB |
| 30B | 9.8 GB | 13.1 GB | 16.9 GB | 20.6 GB | 24.8 GB | 31.9 GB | 60.0 GB |
| 70B | 22.8 GB | 30.6 GB | 39.4 GB | 48.1 GB | 57.8 GB | 74.4 GB | 140.0 GB |
Read it two ways. Across a row, dropping a tier shrinks the file in near-lockstep with the bits. Down a column, every tenfold in parameters is a tenfold in weight size. A 70B at Q4_K_M is about 39 GB; the same tier on 7B is under 4 GB. The table is why quantization, not the raw model, decides what runs locally.
The 30B row sits between the common 13B and 70B footprints and is the scale where a single 24 GB card starts to feel tight at higher tiers. The F16 column is the unquantized reference; everything to its left is a compression of those weights.
Why the numbers are not integers
The bits per parameter are averages, not constants. Q8_0 is 8-bit weights plus one FP16 scale factor per 32 weights, which works out to 8 + 16/32 = 8.5 bits. Q4_0 is likewise 4.5 bits. The K-quant _M variants mix block sizes so medium layers keep extra precision, which is why a tier shows up as a blended figure rather than a flat one. The table uses those blended figures, so a file at Q4_K_M is built from layers at more than and less than 4.5 bits.
Real files run larger
Every size above is a weight-only estimate. Real GGUF files are commonly about 5 to 10 percent larger because llama.cpp keeps the token embedding and the output tensor at higher precision than the body of the model. A 70B at Q4_K_M may therefore land closer to 42 or 43 GB than to 39.4 GB. The exact delta depends on the model’s layer and vocabulary counts.
The standing caveat for any page that uses this table: confirm against the actual downloaded file. The table is the plan; the file is the receipt. This is a site-wide rule, repeated here because this page is referenced more than any other.
Reverse lookup: your VRAM budget
This table starts from a usable VRAM budget and returns the largest model-and-tier combination whose weights fit, weight size only. It assumes the nominal number minus the reserve the driver and engine take, not the box number. Budgets below are usable figures, already reduced from the nominal spec.
| Usable VRAM | Largest fit (weights) | Next step down |
|---|---|---|
| 8 GB | 13B at Q4_K_M (7.3 GB) | 7B at Q8_0 (7.4 GB) |
| 12 GB | 13B at Q6_K (10.7 GB) | 7B at F16 (14.0 GB) is over |
| 16 GB | 13B at Q8_0 (13.8 GB) | 30B at Q4_K_M (16.9 GB) is over |
| 24 GB | 70B at Q2_K (22.8 GB) | 30B at Q5_K_M (20.6 GB) |
| 32 GB | 70B at Q3_K_M (30.6 GB) | 30B at Q8_0 (31.9 GB) |
| 48 GB | 70B at Q4_K_M (39.4 GB) | 70B at Q5_K_M (48.1 GB) is over |
| 80 GB | 70B at Q8_0 (74.4 GB) | 70B at F16 (140 GB) is over |
| 128 GB | still under 70B F16 (140 GB) | split across two pools or accept lower tier |
The reverse lookup is deliberately conservative: it matches weight size to budget and stops. A 24 GB card fits 70B at Q2_K on paper, but Q2_K is the most aggressive tier and the same card would also hold 30B at a much gentler tier with better behavior. The table answers “what fits,” not “what is wise.” For the wisdom, see the graded claims on quality loss.
How to chain this into a real plan
A static lookup answers only the weight half of the question. The full memory need is weights plus KV cache plus runtime overhead. The practical sequence:
- Read the weight size for your model and tier from the table above.
- Subtract the driver and engine reserve from your card’s nominal VRAM to get usable memory.
- Add the KV cache for your planned context from the context length cost page.
- Add a margin for runtime overhead.
- If the sum fits, you are set; if not, drop a tier in the table or pick a smaller model.
The VRAM calculator does steps two through five with live inputs, and the KV cache calculator shows the context-length term on its own. The GPU VRAM reference lists the nominal budgets this reverse table assumes you will discount.
Tiers, briefly
The columns follow the GGUF convention where lower tiers save memory and higher tiers approach the unquantized weights. Q4_K_M is the common default because it keeps most behavior while roughly quartering the 16-bit size; Q8_0 is near-lossless but still compressed; F16 is the reference the others are measured against. The detailed tradeoff, graded by evidence, lives on the quantization comparison page, and the quality side of the same choice is covered by the quantization quality loss page.
What this page does not tell you
- It does not include runtime overhead or KV cache. Those grow with context length and engine, and are the VRAM calculator and KV cache calculator pages’ job, not a static lookup’s.
- It does not cover disk space or download time. A 70B file is large to store and fetch regardless of how fast it loads; plan bandwidth separately.
- It does not judge which tier is best for a task. The quality tradeoff is graded on the quantization quality loss page, and the tier meanings are on the quantization comparison page.
- It does not account for implementation differences between engines. Ollama and llama.cpp may report slightly different footprints for a nominal tier; the Ollama VRAM requirements page covers that engine specifically.
- It does not recommend hardware. The local LLM hardware guide handles budgets without purchase advice.
For the run itself, the run locally hub links the full set, and the 70B local rundown shows which tiers are realistic on common cards. If a model will not fit even at the lowest tier, the Ollama VRAM requirements page covers how an engine reports the remainder.