LLM VRAM Math Cheatsheet

Every VRAM formula for local LLMs on one page: weight size, KV cache, engine overhead, batch multipliers, and worked conversions you can do by hand.

By VRAM Fit editorial Published 2026-09-16 Updated 2026-10-19

This page collects the memory formulas for running models locally in one place. Use it to check a calculator result or to estimate a model on paper. Every line is hand-computable from the model’s published shape.

Weight memory

weight_GB = parameters_billions × quant_bits ÷ 8

The ÷ 8 converts bits per parameter to bytes per parameter. Examples:

7B at 4 bits: 7 × 4 ÷ 8 = 3.5 GB
7B at 8.5 bits (Q8_0): 7 × 8.5 ÷ 8 = 7.44 GB
70B at 4.5 bits (Q4_K_M): 70 × 4.5 ÷ 8 = 39.38 GB

This is the floor. The model will not use less than this, and usually uses more once the cache and engine join.

KV cache

kv_bytes = 2 × layers × kv_heads × head_dim × context × bytes_per_element
kv_GB   = kv_bytes ÷ 1,000,000,000

The leading 2 is the K and V tensors. Per token, drop the context term:

kv_per_token_bytes = 2 × layers × kv_heads × head_dim × bytes_per_element

At 16-bit cache (2 bytes per element) on a 40-layer, 8-KV-head, 128-head-dim model:

2 × 40 × 8 × 128 × 2 = 163,840 bytes per token

Times 32,768 context: 5.37 GB. The context length cost page builds a full ladder from this.

GQA effect

Because kv_heads sits in the formula, cutting it cuts the cache directly. A model with 8 KV heads instead of 32 stores one quarter the KV cache per token. The worked comparison above shows this effect.

Engine and headroom

There is no exact formula; the engine needs memory for the compute context, graphs, and buffers, plus a reserve so the model is not at the limit. A planning estimate of a few gigabytes is typical, scaling with batch size and engine. The GPU reference explains why usable memory is below nominal.

Total memory

total_GB = weight_GB + kv_GB + engine_headroom_GB

Compare total_GB to your card’s usable memory, not its nominal memory.

Batch and concurrency multiplier

kv_total_GB = kv_per_sequence_GB × concurrent_sequences

Weights are shared; caches are not. Serving four chats multiplies the KV term by four. Activations also grow with batch size, adding more on top.

MoE adjustment

For mixture-of-experts models, use the total parameter count in the weight formula. Active parameters change speed, not memory.

Quick conversion reminders

1 byte per parameter at 8 bits; 0.5 byte at 4 bits.
1 GB = 1,000,000,000 bytes for these estimates (decimal, not binary).
To go from bits per parameter to bytes, divide by 8.
To go from GB back to parameters: parameters_billions = weight_GB × 8 ÷ quant_bits.
To sanity-check, recompute the same model at two quants; the difference should match the bit delta times the parameter count.

Worked end-to-end example

Take a 34B model at Q4_K_M (about 4.5 bits), 60 layers, 8 KV heads, 128 head dimension, 16-bit KV cache, and an 8k context on a 24 GB card.

Weight = 34 × 4.5 ÷ 8 = 19.13 GB.
KV per token = 2 × 60 × 8 × 128 × 2 = 245,760 bytes.
KV at 8k = 245,760 × 8,192 ÷ 1,000,000,000 = 2.01 GB.
Engine headroom = 1.5 GB (estimate).
Total = 19.13 + 2.01 + 1.5 = 22.64 GB.
Usable on a 24 GB card after the reserve is about 21 GB, so this does NOT fit comfortably; drop to Q3_K_M or lower context.

Running the same numbers in a calculator should return the same total, which is the check that your hand math is right.

A two-minute estimate

Given a model’s parameters, quant tier, layers, KV heads, head dim, and your context:

Weight = params × bits ÷ 8.
KV = 2 × layers × kv_heads × head_dim × context × bytes ÷ 1e9.
Add a few GB headroom.
Check against usable memory.

If that fits with margin, load it and confirm with a live memory reading. The math here is the plan; the live reading is the proof.

Common model shapes to per-token KV cache

The per-token cache depends only on the attention shape, not on the parameter count. The table below is built directly from published model configs (the num_hidden_layers, num_key_value_heads, and head_dim values). All figures use a 16-bit cache (2 bytes per element) and binary GiB (1 GiB = 2^30 bytes). The per-token column is the unit the KV cache formula produces; the 4K / 32K / 128K columns multiply it by the context length.

Model	Layers	KV heads	Head dim	Per-token KV (FP16)	4K context	32K context	128K context
Llama 3.2 3B	28	8	128	112 KiB	0.44 GiB	3.5 GiB	14 GiB
Llama 3.1 8B / Mistral 7B	32	8	128	128 KiB	0.5 GiB	4 GiB	16 GiB
Qwen 2.5 14B	48	8	128	192 KiB	0.75 GiB	6 GiB	24 GiB
Qwen 2.5 32B	64	8	128	256 KiB	1.0 GiB	8 GiB	32 GiB
Llama 3.3 70B	80	8	128	320 KiB	1.25 GiB	10 GiB	40 GiB

Derivation for the Llama 3.3 70B row: 2 × 80 × 8 × 128 × 2 = 327,680 bytes per token, which is 327,680 ÷ 1024 = 320 KiB. At 4K context that is 320 KiB × 4,096 = 1,310,720 KiB ≈ 1.25 GiB (dividing by 1,073,741,824). The 32K and 128K cells multiply the same per-token number by 32,768 and 131,072 respectively. The shapes are taken from each model’s published config; to verify a specific build, check that model’s config.json. The notable pattern is that going from an 8B to a 70B model multiplies parameters by 8.75 but only raises the per-token KV cache by 2.5, because the cache is driven by layers × KV heads × head dim, not by the parameter count.

A full hand-worked example, step by step

This walks a complete memory estimate for a Llama 3.3 70B at Q4_K_M, 80 layers, 8 KV heads, 128 head dim, a 16-bit cache, and a 32K context. Every number below states where it comes from.

Step 1 — weights. The weight formula is parameters_billions × quant_bits ÷ 8. For an illustrative 70B model at 4.5 effective bits per parameter, 70 × 4.5 ÷ 8 = 39.375 GB, or 36.671 GiB after conversion. Actual tensor mixes and file metadata can change the weight allocation.

Step 2 — KV cache. Use 2 × layers × KV heads × head_dim × context × bytes_per_element. Plugging in the shape gives 2 × 80 × 8 × 128 × 32,768 × 2 = 10,737,418,240 bytes, which is exactly 10 GiB at the 2^30 convention. This matches the 32K column for the 70B row above, and the agreement is the check that the per-token table and the formula are consistent.

Step 3 — overhead. Add a planning estimate of a few gigabytes for the engine context, graphs, and a reserve. At 2 GiB of user-chosen reserve the baseline is 36.671 + 10 + 2 = 48.671 GiB, or about 52.260 GB. Keep the units consistent across all three terms.

Step 4 — compare. Against usable memory, not nominal. Against an actually available 48 GiB budget this baseline exceeds the budget by about 0.671 GiB. A 64 GiB budget leaves about 15.329 GiB before extra effects. Nominal capacity and adding memory across cards do not establish available memory or runtime support. The point of the walk is that the weights and the cache are computed from two independent inputs, then summed, so neither can be inferred from the other.

Unit traps that flip the answer

A correct formula with a wrong unit produces a number that looks plausible and is wrong by a factor. The traps below are the ones that most often change the result by a meaningful margin.

GB versus GiB. 1 GB = 1,000,000,000 bytes; 1 GiB = 1,073,741,824 bytes. The formula sections above use decimal GB; this cache table uses GiB. Divide a GB value by 1.073741824 to convert to GiB before adding it to a GiB value. The calculators state their units explicitly.
Bits versus bytes. Weights are quoted in bits per parameter and the cache in bits per element, but memory is counted in bytes. The ÷ 8 is mandatory: 4-bit weights are 0.5 byte per parameter, 16-bit cache is 2 bytes per element. Forgetting the divide-by-8 overstates memory by 8 times.
Millions versus billions of parameters. A 7B model means 7 billion parameters, entered as 7 in a billions-based formula. Entering it as 7000 (treating the label as millions) overcounts by 1000 times. The model card’s “B” suffix is billions; read it literally.
Context in tokens, not kilobytes. A “32K context” means 32,768 tokens, not 32 KiB of text. The cache formula multiplies by the token count, so substituting a byte or character count for the token count understates the cache by one or two orders of magnitude depending on the text. Count tokens, not characters.

Taking the table offline

The per-token table above is reproducible without any tool. Print or screenshot it, then recompute any cell from four inputs: layers, KV heads, head dim, and bytes per element. The per-token size is 2 × layers × KV heads × head_dim × bytes_per_element, and the cache at a context length is that per-token size times the context, divided by 1,073,741,824 for gibibytes (GiB). With those four numbers written down, every entry in the table can be rebuilt by hand, which is what makes the cheatsheet usable on a machine with no network and no calculator. The same four inputs are what the KV cache calculator asks for, so the offline math and the live tool should return the same figure.

Asked before trying it

What is the one formula I need for weight size?

Weight memory in GB equals parameters in billions times quant bits divided by 8. A 7B model at 4-bit is 7 times 4 divided by 8, or 3.5 GB, before any cache or overhead.

How do I compute KV cache by hand?

Multiply 2 times layers times KV heads times head dimension times context length times bytes per element, then divide by a billion for gigabytes. The 2 is for the separate K and V tensors.

Where does batch size enter the math?

The KV cache multiplies by the number of concurrent sequences, and activations scale with batch size, so total memory rises roughly in proportion to how many sequences you serve at once.

Are these formulas exact?

The weight and KV cache formulas are exact given their inputs. Engine overhead is an estimate, so the final total is a planning figure with a few gigabytes of slack built in.

Do I still need the calculator page?

The [VRAM calculator](/vram-calculator/) applies these same formulas with inputs, so use it for a specific model and use this page as the reference you can check it against.