This calculator answers the question that follows “does the model load”: given the weights, how long a context actually fits in the remaining memory? Enter the model’s attention shape and the VRAM the engine can use, and it returns the per-token cache cost, the room left after the weights, the longest context before the cache alone fills memory, and a fixed-context reference table at 4K, 8K, 32K, and 128K. Input values are calculated locally and are not sent to the operator. The full memory picture, including weights and overhead together, lives in the VRAM calculator.
What the calculator returns
The four numbers in the result answer the same question from four angles.
- Per-token cache cost — the bytes added for every single token of context, from the formula
2 × layers × KV heads × head dim × (cache bits ÷ 8). On a standard 7B GQA shape this is about 128 KiB per token; on a 70B GQA shape it is about 320 KiB per token. The cache scaling is set by the attention shape, not by the parameter count. - Remaining VRAM for the cache — the available memory minus the weights and a fixed overhead. This is the budget the context must live inside.
- Maximum context — the remaining budget divided by the per-token cost. It is the headline number: the longest context that fits with the weights already loaded.
- Fixed-context reference — the cache size at 4K, 8K, 32K, and 128K, each marked as fitting or exceeding. This is the part most people actually check against a model card that advertises a large max context.
The breaking point is reported as the context length at which the cache alone consumes the entire remaining budget. It is a memory-only limit for one dense-attention sequence. A model’s supported context and runtime limits may be lower.
The step table: context length to KV cache to remaining VRAM
The table below is an estimate built from the per-token values in the site facts file. It uses a 7B-class GQA shape (32 layers, 8 KV heads, 128 head dimension, 16-bit cache = 128 KiB per token) on a 24 GiB pool holding 4 GiB of Q4_K_M weights plus about 1 GiB of overhead, leaving 19 GiB for the cache. Numbers are derived, not measured, and depend on the exact quant, engine, and overhead.
| Context length | KV cache (estimate) | Remaining VRAM (estimate) |
|---|---|---|
| 4K | 0.5 GiB | 18.5 GiB |
| 8K | 1.0 GiB | 18 GiB |
| 16K | 2.0 GiB | 17 GiB |
| 32K | 4.0 GiB | 15 GiB |
| 64K | 8.0 GiB | 11 GiB |
| 128K | 16.0 GiB | 3 GiB |
| ~152K | 19 GiB | ~0 GiB (breaks) |
The shape of the curve is the point: the same budget has much less room at 128K than at 4K, with zero change to the weights. The estimates follow the same formula the calculator uses, so the live tool reflects the same numbers for any shape you enter.
Reading the breaking point
The breaking point is not a round number on the model card. It falls wherever the cache size crosses the remaining VRAM. On a 24 GiB card with a 7B at Q4, the cache alone reaches the limit near 152K tokens; on a 16 GiB card the same model breaks far earlier, around 100K. The headline lesson is that context length is a memory budget the operator chooses, not a fixed property of the model. Setting the smallest context that covers the longest real exchange is what keeps a session stable.
Two inputs move the breaking point the most. Lowering the weight quant frees VRAM that becomes cache room, so a 7B at F16 (~13.04 GiB) breaks far sooner than the same model at Q4_K_M (~3.67 GiB). Raising the KV-head count multiplies the cache directly: an MHA 7B with 32 KV heads has four times the per-token cost of a GQA 7B with 8, so it breaks at roughly a quarter of the context. The calculator makes both effects exact for the shape entered.
How this differs from the other tools
The KV cache calculator answers “how big is the cache at a context I pick.” This page answers the inverse: “given my card, what is the largest context the cache can hold after the weights?” The VRAM calculator folds weights, cache, and overhead into one total. The three pages cover the same physics from different directions, and the per-token figure is identical across them because they share one formula.
What this tool does not tell you
- It assumes dense attention with a single shared context per sequence. Hybrid, sliding-window, compressed and recurrent architectures need separate accounting; this is not a validated universal bound for them.
- It does not include batching. Each concurrent sequence carries its own KV cache, so a server multiplies the cache on top of the shared weights. For concurrency, see the batch-size and throughput page.
- It does not model activation peaks during prefill. The largest transient allocation can exceed the steady-state cache; budget extra headroom beyond what the per-token figure implies.
- It uses nominal per-element sizes. Actual allocations vary by tensor mix, runtime and build; use observed weight memory when possible.
- It does not decide a quant or a card. It reports the consequence of the numbers entered; the weight figure should come from the model file size lookup and the usable memory from the GPU reference.
For the theory behind the per-token figure, see KV cache size, explained. For how context length turns into cost in a real session, see context length cost. To check whether a specific model fits a specific card before considering context, open the VRAM calculator. The run locally hub links every page in the site.
Convert units before comparing tools
All memory fields and results on this page use GiB = 1,073,741,824 bytes. The full VRAM estimator uses decimal GB and shows GiB too. Convert decimal GB to GiB by dividing by 1.073741824 before entering a budget here. The 32-layer, 8-KV-head, 128-dimension FP16 shape costs 128 KiB per token in all three tools.
The default is an illustrative 24 GiB available − 4 GiB weights − 1 GiB reserve = 19 GiB cache budget. At 131,072 bytes per token that holds 155,648 tokens before unmodelled effects; this does not mean a model supports that context. No specific GPU performance or hardware fit is guaranteed.