When you plan a local model, the first number you need is how much memory your card exposes. This page collects nominal VRAM for common cards across NVIDIA, AMD, and Apple. The critical caveat follows the table: the printed number is not the number your model receives.
Discrete NVIDIA cards
| Card | Nominal VRAM | Common local-LLM role |
|---|---|---|
| RTX 3060 | 12 GB | Entry 7B/13B at mid quant |
| RTX 4060 Ti | 16 GB | 13B comfortable, 7B with headroom |
| RTX 3090 | 24 GB | Popular 13B/34B, tight 70B |
| RTX 4090 | 24 GB | Fast 13B/34B, tight 70B |
| RTX 5090 | 32 GB | Large 34B, partial 70B |
| RTX 4080 | 16 GB | 13B class |
Discrete AMD cards
| Card | Nominal VRAM | Notes |
|---|---|---|
| RX 6700 XT | 12 GB | ROCm-supported, 7B/13B |
| RX 7900 XTX | 24 GB | Large consumer pool on AMD side |
| RX 7600 | 8 GB | Small models only |
Apple Silicon unified memory
| Tier | Nominal pool | Usable for model (estimate) |
|---|---|---|
| M-series 16 GB | 16 GB | ~11 GB |
| M-series 24 GB | 24 GB | ~18 GB |
| M-series 36 GB | 36 GB | ~28 GB |
| M-series 48 GB | 48 GB | ~39 GB |
| M-series 64 GB | 64 GB | ~54 GB |
| M-series 96 GB | 96 GB | ~84 GB |
| M-series 128 GB | 128 GB | ~115 GB |
The Apple rows show nominal unified memory, not the model ceiling; the system reserve lowers the usable figure further.
Integrated graphics
Most integrated GPUs share system RAM and expose little or no dedicated memory, so they are not a fit for anything past tiny models. If you run on integrated graphics you are effectively using CPU plus system RAM, which means CPU offload.
Nominal is not usable
Every card in the table reserves memory before your model sees it. The driver allocates a context, the compute stack holds buffers, and the inference engine keeps graphs and workspaces. On a 24 GB card the model might realistically get a few gigabytes less than 24. The exact shortfall depends on the driver version and the engine, so plan with a margin rather than to the exact spec.
This gap is why the VRAM calculator asks for available VRAM as a separate input instead of assuming the box number. Enter what the model can actually use, not the marketing figure.
How to use this table
- Find your card’s nominal VRAM.
- Subtract a few gigabytes for the reserve and engine overhead.
- Compare that usable figure to the weight size of your target model from the quantization comparison.
- Add the KV cache for your planned context from the context length cost page.
- If the sum fits with margin, you are set; if not, lower the quant or pick a smaller model.
Laptop and mobile variants differ
A mobile graphics chip with the same name as a desktop card almost always ships less memory and lower power. The RTX 4090 in a laptop is not the same device as the desktop RTX 4090 and exposes a different memory total. When matching a model to a laptop, read the specific mobile SKU’s memory rather than trusting the shared name. The same caution applies to integrated graphics, which borrow from system RAM and should be treated as CPU-offload hardware, not as a fit card.
Specs can drift
Vendor revisions sometimes change memory on the same model name, and mobile variants almost always differ from desktop ones. The values above are the common nominal figures as of writing; confirm the exact card you own against the manufacturer page before buying or planning around a specific number. This table is a planning aid, not a specification guarantee.
What this VRAM actually buys you (estimate)
The table maps a nominal VRAM figure to the largest model that fits by weight at each common quant, using the same weight = parameters × bits per parameter ÷ 8 formula. These are weight-only estimates against the nominal number on the box; the usable figure is lower after the driver and display reserve described below, and the real GGUF file typically runs 5–10% larger than the weight figure, so confirm against the actual file size.
| Nominal VRAM | Q4_K_M | Q5_K_M | Q6_K | Q8_0 | F16 / BF16 |
|---|---|---|---|---|---|
| 8 GB | ~14 B | ~12 B | ~10 B | ~8 B | ~4 B |
| 10 GB | ~18 B | ~15 B | ~12 B | ~9 B | ~5 B |
| 12 GB | ~21 B | ~17 B | ~15 B | ~11 B | ~6 B |
| 16 GB | ~28 B | ~23 B | ~19 B | ~15 B | ~8 B |
| 24 GB | ~43 B | ~35 B | ~29 B | ~23 B | ~12 B |
| 32 GB | ~57 B | ~47 B | ~39 B | ~30 B | ~16 B |
| 40 GB | ~71 B | ~58 B | ~48 B | ~38 B | ~20 B |
| 48 GB | ~85 B | ~70 B | ~58 B | ~45 B | ~24 B |
| 80 GB | ~142 B | ~116 B | ~97 B | ~75 B | ~40 B |
A long context reduces every figure, because the KV cache grows with sequence length and with the model’s layer and head shape rather than with its parameter count. Subtract the driver and display reserve, then add the KV cache for the planned context from the context length cost page; what remains is the model that actually loads. The model file size lookup lists real per-model, per-quant file sizes for a more direct check.
Nominal is not usable: driver and display overhead
The printed VRAM number is a hardware ceiling, not a budget the model can spend. The display output consumes a portion of memory for the framebuffer, and the driver plus the compute stack allocate contexts, buffers, and workspaces before any model is loaded. On a multi-GPU machine each card carries its own fixed overhead, so adding a second card does not double the usable total in a simple way — the second card’s model partition still pays the same base cost. The exact shortfall depends on driver version, engine, and resolution, so plan with a margin rather than to the exact spec. This is the same caution carried in the “Nominal is not usable” discussion above.
Some cards ship in two VRAM versions
A few models are sold in two memory configurations, and the lower-VRAM SKU is easy to confuse with the higher one. Examples include a card offered as both 8 GB and 12 GB, another offered as both 10 GB and 12 GB, and another offered as both 8 GB and 16 GB. The exact split varies by vendor and revision, and the lower-VRAM variant changes which models fit. Always read the specific SKU’s memory on the vendor spec page rather than assuming a model name implies a single memory size; the planning figures above assume the nominal value you confirm there.
Bandwidth decides speed, capacity decides fit
Two different limits govern a local model. Capacity — the amount of VRAM — decides whether the model loads at all: if the weights plus KV cache plus overhead exceed the pool, the load fails. Bandwidth — how fast the GPU moves data — decides how quickly tokens are produced once the model is loaded. A card with more bandwidth generally sustains higher token throughput; a card with more capacity simply holds a bigger model. The two are independent, so a smaller-VRAM card with high bandwidth can feel faster on a model it can hold, while a large-VRAM card may load a bigger model but move data more slowly. Neither property is captured by the nominal VRAM number alone, which is why this page reports capacity and points to the quantization comparison for the size side of the trade.