How to Measure Real VRAM Usage While Running a Model

Three commands to measure actual VRAM while a model runs on NVIDIA, macOS, and Linux, plus a table decoding what each reading means for your fit decision.

By VRAM Fit editorial Published 2026-09-16 Updated 2026-09-17

Estimates tell you whether a model should fit; measurement tells you whether it actually does. This page gives one concrete command per platform and a table for reading what the output means, so you can confirm a fit before trusting it.

NVIDIA: nvidia-smi

While the model is loaded, run:

nvidia-smi

The output lists each GPU with Memory-Usage as used / total in MiB. For a live view during generation, watch it:

watch -n 1 nvidia-smi

Read the used value at the moment of peak context. That is the true memory the model plus engine consumed, including the KV cache it built.

macOS: vm_stat and Activity Monitor

Apple Silicon has no VRAM counter because memory is unified. Check the pool with:

vm_stat

It prints page counts; Pages free, Pages inactive, and Pages occupied by compressor tell you how tight the pool is. For a model pushing past physical memory, watch Pages swapped out climb, which means the system is swapping and generation will stall. Activity Monitor’s Memory tab shows the same in gigabytes and is easier to read at a glance.

Linux without a GPU: free and /proc/meminfo

Under CPU offload the model lives in system RAM. Check it with:

free -h

or watch the resident set of the process:

watch -n 1 "cat /proc/meminfo | head -3"

The Mem line shows used versus total RAM. A model loaded into RAM appears as a large chunk of used memory; if available drops near zero, the machine will swap and slow drastically.

What each reading means

Platform	Command	Reading to watch	What it tells you
NVIDIA	nvidia-smi	Memory-Usage used	Real VRAM at load and peak
NVIDIA	watch nvidia-smi	Used climbing during long replies	KV cache growth under context
macOS	vm_stat	Pages swapped out	Model spilling past unified pool
macOS	Activity Monitor	Memory used by process	Model footprint in the shared pool
Linux	free -h	available	RAM left for CPU-offloaded model
Linux	/proc/meminfo	MemAvailable	Headroom before swapping

Turning measurement into a decision

Load the model at your normal context.
Note the used memory at load. Compare it to the VRAM calculator estimate; a large gap means the engine holds more than you modeled.
Send a long prompt and watch the number rise. The increase is the KV cache, which grows with the context.
If used approaches total, step down a quant tier from the quantization comparison or lower context.
On Apple Silicon, if swapping begins, the model is over the real usable pool, and you should step down a tier.

A worked reading

Suppose nvidia-smi shows a 24 GB card with 21.4 GB used after loading a 13B model at Q4_K_M with an 8k context. The calculator estimated about 7.3 GB of weights plus roughly 1.3 GB of KV cache plus a couple of gigabytes of overhead, which lands near the observed figure. If instead the reading shows 23.8 GB used at load, the model is at the edge and the first long reply will likely fail; step down a quant tier or lower context before trusting it. The live number is the verdict, not the estimate.

Why measure at peak, not load

A model can load inside the limit and then exceed it once the KV cache fills during a long reply. The load reading is the floor, not the ceiling. Always measure during the longest exchange you actually intend to run; that peak is the number your hardware must survive.

When the numbers disagree with the estimate

If measurement shows far more memory than the calculator predicted, the engine is holding buffers or the KV cache is at higher precision than assumed. If measurement shows less, you may have left context headroom unused and can raise it. Either way, trust the live number for the final fit decision and keep the GPU reference handy for comparing against your card’s nominal total.

Estimate versus measured: why the numbers drift

The calculator and the live reading rarely match exactly. The table below lists the common reasons, the direction of the difference, and a rough magnitude. Every magnitude is an estimate that depends on the engine, the model, and the context length.

Factor	Direction (measured vs estimate)	Rough magnitude (estimate)
Real GGUF volume	Measured higher	~5–10% above the weight table, because token embedding and output tensors stay at higher precision
Engine reserve and workspaces	Measured higher	A few gigabytes held by the driver, compute context, and graphs
Context longer than modeled	Measured higher	KV cache scales with sequence length; a 70B model at 32K context uses about eight times the KV cache of the same model at 4K context
Activation and generation buffers	Measured higher	Several gigabytes during generation, outside the static weight figure
Driver and display occupancy	Usable lower	A few gigabytes on discrete cards before the model is loaded

None of these is a fault; they are allocations the formula does not model. The KV cache size explained page shows where the context term comes from, and the VRAM math cheatsheet lists the formula side by side with the live terms.

Change one variable, then measure again

A reproducible check changes a single input and watches one reading:

Change the quant. Load the same model at Q4_K_M, then at Q8_0. Watch the load-time used memory; the gap should track the bit-width difference from the quantization comparison. This isolates weight size from everything else.
Change the context length. Keep the quant fixed and reload at a short context, then at a long one. The rise in used during generation is the KV cache, and its size should follow the model’s layer and head shape from the KV reference.
Change the concurrency. Serving multiple sequences shares the KV cache per sequence, so repeating the load with more parallel requests grows memory roughly in proportion. Watch the peak, not the load reading.

Measuring one variable at a time keeps the cause of any change identifiable.

What the reading does not charge to the model

Not everything in a memory reading belongs to the model. On macOS, the window server and desktop compositing draw from the same pool and show up in the total even though they are not model memory. Other running processes — a browser with many tabs, background sync, an editor — occupy part of the pool and reduce what the model can take. On systems using shared memory for display, a portion of RAM is reserved by the display stack and is not available to the model even when no model is loaded. When reading a number, separate the model’s own allocation from these shared and background consumers; only the model’s allocation should drive the fit decision.

When to trust the measured number over the estimate

The estimate is the right tool for planning before anything is loaded, because it is fast and needs no hardware. The live measurement is the right tool for the final decision, because it captures the engine’s real allocations that the formula omits. Trust the measured number when: the model is at or near the limit, the context is long, the quant is unusual, or the engine is one the calculator was not tuned for. In those cases the live peak reading is the verdict, and the estimate becomes a cross-check rather than the authority. The GPU reference is useful for comparing the measured peak against the card’s nominal total.

Asked before trying it

What command shows VRAM on an NVIDIA card?

nvidia-smi reports total and used memory for each GPU. Run it while the model loads and during generation to see the real peak, not just the static weight size.

How do I check memory on an Apple Silicon Mac?

Read Apple Silicon memory from Activity Monitor's Memory tab, or from vm_stat in a terminal. The number that matters is not the total pool but whether the compressed and swapped counters climb while the model runs: if they do, the model is spilling past physical memory and generation will slow sharply.

What about a plain Linux machine without a GPU?

Use free -h or watch /proc/meminfo to see RAM use as the model loads into system memory under CPU offload. The model shows up as a large resident process, not as VRAM.

Why measure instead of trusting the calculator?

The calculator estimates weights plus KV cache plus overhead. Measurement shows the engine's real allocations, including buffers it did not advertise. The two together confirm a fit.

When should I measure?

At load and again at your longest planned context. Usage climbs as the KV cache grows, so a model that looks fine at load can exceed the limit mid-session.