KV Cache Calculator: Context Memory in GB and GiB

A working KV cache calculator: enter layers, KV heads, head dimension, context length and cache precision to estimate the cache in GB and GiB, including concurrent sequences.

Jump to the tool

By VRAM Fit editorial Published 2026-09-17 Updated 2026-10-19

KV CACHE MEMORY ESTIMATOR

Transformer layers KV heads (not query heads) Head dimension Context length in tokens Cache precision in bits Concurrent sequences (whole number)

Per sequence: 1.074 GB (1.000 GiB)
Concurrent sequences: 1 → 1.074 GB (1.000 GiB)
Per token, all layers: 128.00 KiB
Formula: 2 × 32 layers × 8 KV heads × 128 head dim × 8192 tokens × 16/8 bytes × 1 sequences
Units: GB = 1,000,000,000 bytes; GiB = 1,073,741,824 bytes; KiB = 1,024 bytes.
Planning estimate; excludes weights, activations, cache metadata and runtime buffers.

This page computes the KV cache — the part of LLM memory that scales with context length — directly in your browser. Enter the model’s attention shape and your context length, and it returns the cache size in decimal GB and binary GiB, with and without batching. Calculator inputs are processed locally and are not sent to the operator. See the privacy policy for hosting and site services. For weights plus cache plus overhead in one shot, use the full VRAM calculator.

Why the cache deserves its own calculator

The full memory estimator answers “does the model fit”; this one answers the follow-up question that bites people after the model already loads: why did it run out of memory the moment the context got long? Weights are a constant; the cache is a function of context. For this page’s 32-layer GQA shape, a 1,024-token FP16 cache is 0.125 GiB (0.134 GB), while 131,072 tokens require 16 GiB (17.180 GB). The weights remain unchanged. The calculator above makes that trade-off exact for your model’s shape.

Three levers shrink the cache, in order of practicality:

GQA / MQA head count — already baked into the model. Fewer KV heads means a proportionally smaller cache; this is why modern models ship with 8 KV heads instead of 32. You cannot change it, but you should enter the KV head count here, not the attention head count.
Cache precision — halving 16-bit to 8-bit halves the cache. Only trust this if your engine documents cache quantization and you have confirmed it is active; see the full estimator’s FAQ on why cache and weight bits are separate fields.
Context length — the direct multiplier. Halving context halves the cache, which is often the honest answer when a long-context workload does not fit.

Worked examples

Model shape	Layers	KV heads	Head dim	Context	Cache (FP16)
7B-class (GQA)	32	8	128	8,192	1.074 GB / 1 GiB
7B-class (MHA)	32	32	128	8,192	4.295 GB / 4 GiB
70B-class (GQA)	80	8	128	32,768	10.737 GB / 10 GiB
70B-class (GQA)	80	8	128	131,072	42.950 GB / 40 GiB

The MHA row is the lesson: identical model size, four times the cache, purely from the head layout. That gap is why two “7B” models can behave completely differently at long context.

How to read the result

The result block returns four things, and each answers a different question.

Per sequence (GB and GiB). The KV cache for a single sequence at the context length you entered. This is the number to compare against the room left after the weights on your card.
Concurrent sequences. When the batch field is greater than one, the tool multiplies the per-sequence cache by the number of concurrent sequences. Each live conversation carries its own cache, so this is the figure a server actually pays.
Per token, all layers (KiB). The fixed cache added for every single token, computed as 2 × layers × KV heads × head dim × (cache bits ÷ 8) ÷ 1024. For 32 layers with 8 KV heads, head dimension 128 and FP16 storage, that is 128 KiB per token across the complete model.
Formula and units. The output shows every multiplier and states the byte convention; it does not claim a particular GPU will fit the model.

The result is a planning estimate from the inputs, not a measurement of a running engine; the timestamp in the output says so explicitly.

What the batch multiplier means

The KV cache is stored per sequence in flight. Serving four users at once therefore stores four caches on top of the one shared set of weights. The batch field captures exactly that: it scales the per-sequence cache by the number of concurrent sequences and leaves the weight term untouched.

This is why concurrency is the hidden memory cost of self-hosting. A single-user run can look comfortable while a server with several live chats silently multiplies the cache past the card. The multiplier here is a straight product — no engineering overhead is added — so the batch figure is a baseline for the concurrent independent sequences you enter; shared prefixes, allocation policy and buffers can change actual usage. The batch size and throughput page extends the same idea to activations, which also rise with batch size.

Unit and precision traps

A few inputs are easy to get wrong, and each flips the result.

Gigabytes and gibibytes. 1 GB = 1,000,000,000 bytes; 1 GiB = 1,073,741,824 bytes. The same allocation therefore has a larger numerical value in GB. The tool shows both so you can compare it with the full VRAM estimator without mixing units.
Cache precision is separate from weight quant. The cache bits field defaults to 16. Most engines keep the cache at 16-bit even when the weights are 4-bit, so a 4-bit model can still carry a full-precision cache. Lowering the cache bits to 8 roughly halves the cache term, but only if the engine supports cache quantization and you have confirmed it is active.
Enter the KV head count, not the attention head count. Grouped-query attention shares one key and value head across several query heads, so only the KV-head number enters the formula. A 7B with 8 KV heads and one with 32 KV heads (multi-head attention) differ by 4x in cache from the head layout alone; entering the total attention heads overstates the cache by that factor.
Context is in tokens, not kilobytes. The context field is the token count (4096, 32768, 131072). Substituting a character or byte count understates the cache by one or two orders of magnitude.

For the inverse question — given a card, what is the longest context that fits after the weights — the context length calculator turns the same per-token figure around and reports the breaking point directly.

What this tool does not do

It does not estimate weights, activations, or engine overhead — that is the full VRAM calculator’s job.
It assumes dense attention with a single shared context per sequence. Sliding-window, hybrid or recurrent architectures require their own accounting; this is not a validated universal bound for them.
Prefix caching and paged-attention engines (vLLM, llama.cpp server) manage the cache dynamically across requests; the per-sequence number here is the unit they budget with, not the total they reserve.

For the theory behind the formula, see KV cache size, explained. For how the cache interacts with quantization choices, see quantization formats compared.

Check the default by hand

At 32 layers, 8 KV heads, head dimension 128 and FP16 storage, the cache adds 131,072 bytes = 128 KiB per token across all layers. At 8,192 tokens that is 1,073,741,824 bytes = 1 GiB = 1.073741824 GB for one sequence. Four independent sequences use 4 GiB = 4.294967296 GB before metadata and runtime buffers. Halving context or cache precision halves this baseline; changing weight quantization alone does not.

The geometry follows the llama.cpp maintainer explanation. The full LLM VRAM calculator uses the same cache function and adds weights plus an editable runtime reserve.

Keep this setup

Save the settings to your Sitefarm account. Uploaded files and generated results are not stored.

Asked before trying it

What is the KV cache formula?

KV cache bytes = 2 × layers × KV heads × head dimension × context length × (cache bits ÷ 8). The 2 accounts for keys and values; everything else you enter below. The calculator also shows a batch-size multiplier because the cache is per sequence in flight.

Why does the KV cache grow with context length but weights do not?

Weights are fixed the moment the model file is built — a 7B at Q4_K_M weighs the same whether your context is 1K or 128K. The cache holds keys and values for every token in the context, so doubling the context doubles the cache. At long context the cache, not the weights, is what runs you out of memory.

What is GQA and why does it shrink the cache?

Grouped-query attention shares one key/value head across several query heads, so only the KV-head count enters the cache formula. A model with 8 KV heads instead of 32 has a cache four times smaller. The KV-head count is the number to enter here, not the total attention heads.

Does quantizing the model quantize the KV cache too?

No. Weight quantization and cache precision are independent. Most engines keep the cache at 16-bit unless you explicitly enable a cache quantization option, so a 4-bit model can still carry a full-precision cache.

How do I find my model's layer and KV-head counts?

The model card or config.json lists num_hidden_layers, num_key_value_heads, and head_dim (or derive head dim from hidden_size ÷ attention heads). The presets below cover common shapes so you do not have to look them up every time.