The question “which GPU for a local LLM” is really two questions: will the model fit, and will it run at a usable pace. Capacity is the first gate and the easiest to check; bandwidth and format support decide whether the fit is practical. This page maps VRAM brackets to model scale per quant tier, then covers the specs beyond raw capacity. It does not recommend a specific brand or model to buy. The choice belongs to the reader; the page only supplies the capacity and bandwidth criteria to make that choice.
VRAM bracket to model scale, by quant tier
The table pairs a single-card VRAM bracket with the model scale it can hold as a resident load, at two quant tiers. Weight baselines come from the site’s GGUF size table: 7B is about 3.9 GB at Q4_K_M and 14.0 GB at BF16, 13B about 7.3 GB at Q4_K_M, 30B-class about 16 to 19 GB at Q4_K_M, and 70B about 39.4 GB at Q4_K_M and 74.4 GB at Q8_0. These are planning estimates; the real file size should be confirmed before download.
| VRAM bracket | Holds at Q4_K_M (estimate) | Holds at higher precision (estimate) | Suits |
|---|---|---|---|
| 8 GB | 7B-class with headroom | Below 7B at BF16 | Small chat, single 7B Q4 assistant |
| 12 to 16 GB | 7B to 13B Q4 | 7B at BF16 on the 16 GB end | Everyday 7B or 13B Q4 chat |
| 24 GB | 13B to ~30B Q4 | 7B at BF16 | 13B Q4 with context, small full-precision work |
| 32 GB | up to ~30 to 34B Q4 | 13B at BF16 | Mid-scale Q4 with room for context |
| 48 GB | 70B at Q4 with headroom | 13B to 34B at BF16 | 70B Q4 local, larger context budgets |
| 80 GB | 70B at Q4 or Q8 | 70B near Q8 | 70B near-lossless, large context and batch |
The “suits” column is a capacity judgment, not a purchase ranking. The relevant quant tiers and their weight math are detailed in the quantization formats compared page, and exact file sizes live in the model file size lookup.
One caveat sits inside every bracket above: the weight figure is only the floor. A running session also holds a KV cache that grows with context length, and activations that grow with batch size. A 24 GB bracket that holds a 13B model at Q4 leaves far less room once a long document and several concurrent chats are in flight, because the cache is allocated per live sequence on top of the weights. The context length cost page quantifies that extra; the bracket table here assumes a modest context and should be read as the load-only ceiling, not the served-with-headroom figure. When a model sits close to the bracket edge, treat the next bracket up as the realistic choice rather than the exact one.
Three specs beyond raw capacity
VRAM is necessary but not sufficient. Three more specs decide whether a card that fits a model also runs it acceptably.
- Memory bandwidth sets the speed. Generation is memory-bound: each token reads the full weight set, so throughput tracks bandwidth more than compute. A larger but slower card may load a bigger model yet answer more slowly. This page deliberately gives no tokens-per-second figures, because they shift with the build, driver, and offload ratio.
- Memory bus width. The bus width is the pipe behind the bandwidth number. A wider bus generally sustains higher bandwidth, which is why two cards with the same VRAM can feel different in practice. Read it alongside the bandwidth, not instead of it.
- Operator and quant-format support. A card may have the bytes but lack the kernel for a format, forcing a fallback that will not run or runs far slower. Confirm that the target quant tier and any special operators are supported before treating a model as runnable. This is the most common reason a model that fits on paper still fails to launch, and it is invisible from the VRAM number alone. The LLM engine comparison page shows how engine support for a format varies, which is the layer where this limit actually appears.
These three specs interact. A wide bus with low bandwidth support for the needed format is still a poor fit, and a huge VRAM pool on a narrow bus may hold a model it cannot serve quickly. Capacity, bandwidth, and format support are three independent gates, and a model must clear all three to be practical, not merely possible.
Unified memory versus discrete VRAM
Discrete GPUs carry their own VRAM, separate from system RAM. Apple Silicon and similar designs use unified memory, a single pool shared by CPU and GPU. The practical effect: a large unified-memory tier can hold a model that no single discrete card could, because the pool is bigger than any discrete VRAM bracket listed above. The cost is bandwidth and operator support, which follow different rules than discrete GPUs. The Apple Silicon unified memory page covers that trade in detail, and the run locally hub shows how the two paths branch.
Nominal VRAM is not usable VRAM
The number on the box is the nominal capacity, not what the model receives. On Windows and Linux, the display output and driver reserve part of the memory; on macOS, part of unified memory is likewise held back by the system. The usable pool is below nominal, so a bracket in the table above should be read against the usable amount, not the sticker. When a model that “should fit” overflows, the reserved slice is the first thing to check; the OOM guide walks through that diagnosis.
This page does not recommend a specific brand or model to purchase. The choice is the reader’s. The page supplies only the capacity and bandwidth criteria: pick the smallest bracket that holds the target model at the target quant tier with headroom, then weigh bandwidth for the speed the task needs.
When no single bracket clears all three gates, splitting the load across cards is the alternative path rather than a single bigger purchase. The multi-GPU VRAM page covers how weights and caches distribute across devices, which changes the capacity math from a single bracket into a pooled one. That is a separate decision from the per-card sizing this page addresses, and it does not change the bandwidth and format gates a model must still meet.
What this page does not tell you
- It does not give measured speeds. Real generation rate depends on the build, driver, batch, and offload ratio, none of which this page estimates.
- It does not cover used-market pricing. Resale values move constantly and are outside the capacity framing here.
- It does not assess full-system compatibility. Power supply, chassis, cooling, and platform support decide whether a card installs, which is a separate question from whether it fits a model.
For exact per-model memory and a headroom check against a real card, use the VRAM calculator and the Ollama VRAM requirements reference rather than the brackets above.