Apple Silicon Macs are attractive for local models because the memory is unified: the CPU, GPU, and neural engine share one pool. That removes the copy step between system RAM and VRAM that slows discrete-GPU setups. The catch is that the operating system wants part of that pool too, so the number printed on the spec sheet is not the number your model gets.
Unified memory versus discrete VRAM
On a PC with a graphics card, the GPU has its own dedicated memory and the CPU has separate RAM. A model must fit entirely in the GPU’s memory to run fast, which is why a 24 GB card caps you hard. On Apple Silicon, there is no separate GPU memory; the GPU reads from the same pool as everything else. A 70B model that would never fit one card can load as a single allocation on a 128 GB Mac.
This is the central reason the run 70B locally page lists unified-memory Macs as a valid path. The trade is peak speed: a top discrete GPU still computes faster per token than the integrated GPU in a Mac, but the Mac can simply hold bigger models.
How much you can really give a model
macOS reserves memory for the system, window server, and background processes. The exact reserve varies with the OS version and what else is open, so treat these as planning figures rather than guarantees.
| Nominal unified memory | Rough usable for model | Largest useful model (Q4_K_M) |
|---|---|---|
| 16 GB | ~11 GB | 7B, tight |
| 24 GB | ~18 GB | 13B |
| 32 GB | ~25 GB | 13B comfortable |
| 36 GB | ~28 GB | 13B, room for long context |
| 48 GB | ~39 GB | 34B, not 70B |
| 64 GB | ~54 GB | 70B, tight |
| 96 GB | ~84 GB | 70B comfortable |
| 128 GB | ~115 GB | 70B at higher quant, or 120B-class |
The “usable” column is an estimate; close other apps and you approach it, open a browser with many tabs and you fall short. Activity Monitor’s Memory tab shows the real free memory on macOS.
Why you must not give it all to the model
Two reasons keep the model below the nominal total:
- System reserve. macOS itself needs several gigabytes just to stay alive. Starve it and the machine swaps or the load is killed.
- Headroom for the KV cache. Long context grows the KV cache, as shown on the context length cost page. If you load a model that fills the pool exactly, the first long reply can push past the limit and crash.
A safe rule is to leave at least 10 to 15 percent of the pool free after the weights. On 64 GB, that means planning around a 54 GB model ceiling, which is why 70B at Q4_K_M (about 42 GB) fits but 70B at Q6_K (about 60 GB) does not.
Picking a tier
- For chat with 7B or 13B models, 24–32 GB is plenty and cheap.
- For 34B-class models at a good quant, 48 GB is the practical floor.
- For a real 70B workflow, 64 GB is the minimum and 96 GB is where it stops feeling constrained.
- For 120B-class or high-quant 70B, only 128 GB gives room.
The quantization comparison shows how much each tier saves, which is how you stretch a given Mac tier to a bigger model.
Theoretical capacity by unified-memory tier and quant (estimate)
The table below shows the largest model that fits by weight alone at each common quant, derived from the formula weight (GB) = parameters (billions) × bits per parameter ÷ 8 using the bit widths in the quant reference. These are theoretical maxima against the nominal pool and assume a short context. They are estimates: the real GGUF file typically runs 5–10% larger than the weight figure, and macOS keeps part of the pool for itself, so the usable figure is closer to the reduced numbers in the table above.
| Nominal pool | Q4_K_M | Q5_K_M | Q6_K | Q8_0 | F16 / BF16 |
|---|---|---|---|---|---|
| 16 GB | ~14 B | ~12 B | ~10 B | ~8 B | ~4 B |
| 24 GB | ~21 B | ~17 B | ~15 B | ~11 B | ~6 B |
| 32 GB | ~28 B | ~23 B | ~19 B | ~15 B | ~8 B |
| 36 GB | ~32 B | ~26 B | ~22 B | ~17 B | ~9 B |
| 48 GB | ~43 B | ~35 B | ~29 B | ~23 B | ~12 B |
| 64 GB | ~57 B | ~47 B | ~39 B | ~30 B | ~16 B |
| 96 GB | ~85 B | ~70 B | ~58 B | ~45 B | ~24 B |
| 128 GB | ~114 B | ~93 B | ~78 B | ~60 B | ~32 B |
A long context shrinks these numbers. The KV cache grows with the sequence length and with the model’s layer and head shape, not with its parameter count, so a 70B model at a 128K context can need tens of gigabytes of KV cache on top of the weights. Treat the table as the ceiling for a short-context run, and subtract both the macOS reserve and the KV cache for the planned context from the context length cost page. Confirm against the actual size of the specific GGUF file, which the weight table understates by roughly 5–10%.
The wired limit and the system reserve
The reason a model never receives the full pool is the wired memory limit. macOS keeps a portion of physical memory wired for the system, the window server, and background processes, and only a part of the remainder is offered to the GPU for model allocations. The split can be adjusted through the iogpu.wired_limit_mb setting, but the default does not hand the entire pool to the GPU, and lowering the wired limit too far makes the system unstable or forces swapping. Whether the model gets a comfortable share depends on the OS version, how many other apps are open, and how much memory the window server is holding at the time. None of those values is fixed, which is why the usable figure is always described as an estimate rather than a guarantee. The practical check is to watch Activity Monitor while the model loads: if the free memory after allocation is thin, the model is near the real ceiling regardless of the nominal number.
Capacity versus bandwidth: what unified memory trades
Unified memory is most usefully read as a capacity win, not a speed win. Because the CPU, GPU, and neural engine share one pool, a Mac can hold a model that would need an expensive multi-GPU PC to fit on a single discrete card. The trade is bandwidth: the integrated GPU moves data through a shared bus that a top discrete card with dedicated, higher-bandwidth memory can outpace. The difference is qualitative rather than a fixed number — a discrete card with enough VRAM will generally sustain higher token throughput, while the Mac’s advantage is simply being able to load the larger model at all. The VRAM calculator and the quantization comparison help decide which side of that trade fits a given workload.
What to move first when the model does not fit
When a target model will not load, the order of adjustments matters because each one costs a different amount of quality or speed:
- Lower the quant first. Moving from Q8_0 to Q4_K_M cuts the weight footprint by more than half per the bit-width table, which is usually the single largest saving available.
- Shorten the context. The KV cache scales with sequence length and with the model’s layer and head shape, so trimming context length reclaims memory that the weights do not.
- Offload layers to the CPU. Partial CPU offload trades speed for fit and is covered on the llama.cpp offload layers page; it is a middle step before giving up on the model entirely.
- Free the system reserve. Closing other apps and browser tabs recovers part of the wired and background memory, pushing the usable ceiling back up.
- Step to a smaller model only as the last resort, once the quant, context, and offload options are exhausted.
Each step is reversible and can be tried in order without changing hardware.
Verifying on your own machine
Do not trust the spec sheet alone. Before loading a big model, check Activity Monitor’s Memory tab or run the macOS memory command. If the “memory used” plus your planned model size is near the pool total, step down a quant tier or close apps. The unified pool is generous, but it is still a budget.