LLM VRAM Calculator: Weights, KV Cache & GPU Fit

Estimate LLM GPU memory from weights, KV cache and runtime reserve. Compare GB and GiB, adjust context length and see the full calculation.

Jump to the tool

By VRAM Fit editorial Published 2026-09-16 Updated 2026-10-19

MODEL MEMORY ESTIMATOR

Default example: a 7B dense-attention model needs about 5.97 GB (5.56 GiB), including the chosen 1.5 GB runtime reserve. This is a planning estimate, not a measured hardware result.

Default planning budget: 24 decimal GB. Select your GPU budget or enter memory actually available to the model.

Parameters in billions (required) — 7 for a 7B model, 70 for a 70B model Weight quantization, bits per parameter (required) — 4.5 for Q4_K_M, 8.5 for Q8_0 Memory available to the model (required) — pick a card or type a number

Changing parameter count does not change the attention shape. Use a shape preset or enter your model configuration in Advanced parameters.

Advanced parameters — the model's shape and the runtime budget

Leave these alone unless you know the model's architecture. Every field here has a default that matches a mid-size GQA model at a moderate context, and the result updates as you type.

Transformer layers — 32 for a 7B, 80 for a 70B KV heads (GQA count) — 8 on most modern models, 1 for MQA, the full head count for MHA Head dimension — commonly 128 Context length in tokens — 4096, 32768, 131072 KV cache precision in bits — 16 for the standard FP16 cache, 8 only if you enabled cache quantization Engine headroom in GB — driver context, graph buffers, activations

Shareable link: add parameters to the address, for example /vram-calculator/?params=70&bits=4.5&vram=24&layers=80&kvheads=8&headdim=128&ctx=8192. Any value in the link overrides the detected one.

Weight memory:      3.94 GB
KV cache:           0.537 GB   (128 KiB per token)
Engine headroom:    1.50 GB
Total needed:       5.97 GB
Available:          24.00 GB
Utilization:        24.9%
Verdict:            FITS
Total in binary units: 5.56 GiB
Units: GB = 1,000,000,000 bytes; GiB = 1,073,741,824 bytes; KiB = 1,024 bytes.
Planning estimate for one dense-attention sequence. Check actual free memory and runtime buffers.

Use this LLM VRAM calculator to plan one dense-attention sequence: model weights, KV cache and engine headroom, each as its own line. It computes a result the moment the page loads, using a 7B model at Q4_K_M and a 4K context as the starting point, and it tries to recognise your GPU so the available-memory field is already filled in. Calculator inputs are processed locally and are not sent to the operator. See the privacy policy for hosting and site services.

If you only need the cache and not the weights, the KV cache calculator is the smaller tool. For the reference table of card memory, see GPU VRAM reference.

What the three lines mean

The result panel keeps three quantities apart on purpose, because they behave differently.

Line	Where it comes from	What changes it
Weights	`parameters x weight bits / 8`	The model and the quantization level. Nothing else.
KV cache	`2 x layers x KV heads x head dim x context x cache bits / 8`	Context length and the model’s attention layout. Not the parameter count.
Engine headroom	Entered by you	The inference engine, the driver, and how much you batch.

Two facts make this layout worth the extra lines:

Weights are constant, the cache is not. A 7B model at Q4_K_M weighs roughly 4 GB whether you serve 4K or 128K tokens. Its cache goes from 0.5 GiB (0.54 GB) to 16 GiB (17.18 GB) across that range. A single-multiplier estimate cannot show you that, and the shift is exactly what breaks a configuration that used to work.
Weight bits and cache bits are independent. Quantizing the weights does not quantize the cache. Unless your engine documents a cache quantization option and you have confirmed it is on, leave the cache field at 16.

Worked example: a 7B model at Q4_K_M on a 24 GB card

Defaults on this page: params=7, bits=4.5, kvbits=16, layers=32, kvheads=8, headdim=128, ctx=4096, vram=24, over=1.5.

Weights = 7 x 4.5 / 8 = 3.94 GB
Cache per token = 2 x 32 x 8 x 128 x 2 bytes = 131,072 bytes = 128 KiB per token
Cache at 4,096 tokens = 0.54 GB
Total from unrounded values = 3.9375 + 0.536870912 + 1.5 = 5.974370912 GB ≈ 5.97 GB → fits in 24 GB with 18.03 GB spare.

The per-token figure is the one worth remembering, because it turns any context length into an instant estimate: at 128 KiB per token, 32K context is 4 GiB (4.29 GB) and 128K context is 16 GiB (17.18 GB), for a model whose weights stay under 4 GB.

Worked example: the same model, four times the cache, no change in size

Switch the preset to the MHA shape (layers=40, kvheads=40) on a 13B model and compare it with a GQA model of similar size. Fewer KV heads means a proportionally smaller cache: 8 KV heads instead of 32 cuts the per-token cost by four with no change to the parameter count. That is why two models of the same size can behave completely differently at long context, and why the KV-head field, not the attention-head field, is the one to fill in here. The KV cache explanation covers the geometry; the KV cache calculator isolates the cache if you want to iterate on context length alone.

Worked example: 70B at Q4_K_M

Preset 70B / Q4_K_M / 8K on the same 24 GB card.

Weights = 70 x 4.5 / 8 = 39.38 GB
Cache per token = 2 x 80 x 8 x 128 x 2 bytes = 327,680 bytes = 320 KiB per token
Cache at 8,192 tokens = 2.68 GB
Total = 39.38 + 2.68 + 1.5 = 43.56 GB → does not fit in 24 GB.

This particular 4.5-bit, 8K configuration does not fit the entered 24 GB budget. Lower-bit files can change the estimate, with quality and engine-support trade-offs; this tool does not validate a specific download. The realistic routes are a second card, system-memory offload with the speed penalty that implies, or a smaller model. The 70B local rundown works through the combinations; multi-GPU memory covers how the split is budgeted.

What this calculator does not tell you

It does not predict tokens per second. Speed depends on the compiler build, driver, batch size, and how many layers are offloaded. None of that is modelled here.
It assumes dense attention and a linear cache. Sliding-window, hybrid, compressed and recurrent architectures need architecture-specific accounting. This formula is not a validated estimate or universal upper bound for those models.
It models one sequence. Concurrent requests multiply the cache. Use the batch size guide when sizing a server rather than a workstation.
It uses nominal memory. The figure on the box is not what the model gets. Windows and Linux reserve some for display and driver work, and macOS keeps part of unified memory for itself, so subtract a reserve before trusting a fit.
It ignores activations. Peak memory during prefill can exceed the steady-state total, most visibly with long prompts and large batches.
It cannot see your actual card. The detected value comes from a browser-reported model name mapped through a reference table, not from reading the hardware, and several GPUs ship in more than one memory size.

Common input mistakes

Parameters in millions instead of billions. Entering 7000 instead of 7 inflates the weights a thousandfold.
Using the attention-head count as the KV-head count. On GQA models that overstates the cache, often by four times.
Assuming the card gives the model all of its memory. A 24 GB card does not hand 24 GB to the model.
Typing 4.0 for Q4_K_M. The common approximation is 4.5 bits per parameter; the quantization comparison lists the rest.
Expecting the result to match a real run exactly. Actual file sizes vary with the tensor mix, and the engine adds its own buffers. Treat a fit with under a gigabyte of headroom as tight.

Every formula used here is printed on the VRAM math cheatsheet so the arithmetic can be reproduced without this page. If a number here disagrees with your own calculation, the measurement guide shows how to read real usage out of the running model and compare it against the estimate.

GB and GiB: use the same unit before comparing

The main result uses decimal GB = 1,000,000,000 bytes. Binary GiB = 1,073,741,824 bytes is shown alongside the total; KiB = 1,024 bytes is used for the per-token cache. If a runtime reports free memory in GiB, multiply by 1.073741824 before entering it as GB here. GPU presets and browser-name detection supply planning figures, not measured free memory.

For the default attention shape, the KV cache calculator at 4,096 tokens returns 0.537 GB / 0.500 GiB, matching this tool. Its default is 8,192 tokens, so halve that context before comparing. The complete 7B example is 5.974 GB / 5.564 GiB, with the user-entered reserve included.

The cache shape follows the llama.cpp maintainers’ explanation of KV dimensions. Verify your own model’s configuration; a parameter-count preset does not identify or certify a particular model architecture.

Keep this setup

Save the settings to your Sitefarm account. Uploaded files and generated results are not stored.

Asked before trying it

Does this calculator include the KV cache for long context?

Yes. It computes the cache from the layer count, KV-head count, head dimension, context length and cache precision, and prints it as its own line. Double the context length and the cache line doubles.

Why does the total exceed parameters times quant bits divided by eight?

Weights are only part of the picture. The cache grows with context length, and the engine needs memory for the driver context, graph buffers and activations. The reserve is entered by you, not inferred or measured. The result panel keeps those three parts separate so you can see which one is driving the total.

Should I enter GGUF bits like 4.5 for Q4_K_M?

Enter the approximate bits per parameter for your file. A common approximation for Q4_K_M is 4.5. Effective bits vary by model and tensor mix. Prefer the actual file size when available rather than treating the label as an exact size.

Why is the KV cache precision a separate field from the weight bits?

Because quantizing weights does not quantize the cache. Most engines keep the cache at 16-bit whichever GGUF file you load, so a 4-bit model can still carry a full-precision cache that outgrows the weights at long context. Set 8 only if your engine exposes a KV cache quantization option and you have confirmed it is active.

Can I trust the result to decide a purchase?

Treat it as a planning estimate, not a guarantee. Available memory is always below the nominal number on the box, and engine overhead varies with the runtime and batch size. Read the limitations section before committing.

Does the calculator handle Apple Silicon unified memory?

You can enter the nominal unified memory as the available figure, but macOS keeps part of that pool for itself. The Apple Silicon page covers how much is realistically left for a model, and how the wired limit changes it.