The same model can report a different VRAM footprint depending on which runner loads it. That difference is rarely about the weights, which stay the same size, and almost always about how the engine reserves, allocates, and manages memory around them. This page compares llama.cpp, Ollama, and vLLM on the dimensions that decide whether a model fits and how many users it can serve at once.
The three runners at a glance
llama.cpp is a low-level inference library and command-line tool focused on local, single-user use. Ollama is a model manager that wraps llama.cpp and makes local running and pulling models simpler. vLLM is a serving system designed for concurrent, multi-user request handling through an API. They solve different problems, and their memory behavior follows from those problems.
The table below is the page’s core comparison. It is deliberately qualitative: exact numbers depend on build flags, version, context length, and request pattern, so any fixed figure would mislead. The goal is to show the shape of each engine’s memory behavior, not a measurement.
| Engine | VRAM overhead character | Startup behavior | Concurrency capability | KV cache management | Fits which scenario |
|---|---|---|---|---|---|
| llama.cpp | Small base overhead; memory grows with layers offloaded and context used | Loads the model, offloads layers to GPU by setting, stays idle-cheap | Built for one session at a time; weak for many simultaneous users | Allocated per context as the session needs it; no large reserved pool | One person, local desktop, single chat |
| Ollama | Close to llama.cpp base, plus a thin manager process overhead | Pulls and serves a model on a local port; similar idle profile | Single-user oriented by default; not built to fan out to many users | Inherits llama.cpp-style per-session allocation | One person or one machine, easy local use |
| vLLM | Higher static footprint; reserves a pool of VRAM up front for serving | Starts a server, pre-allocates scheduler and cache space | Strong concurrent handling of many requests through batching | Paged KV cache in reserved blocks, grown and reused per request | Several users, an API service, shared endpoint |
Why the same model shows different memory under different engines
Model weights are a fixed cost: a Q4_K_M file of a given size takes the same bytes whether llama.cpp or vLLM loads it. The variation comes from everything around the weights.
A single-user runner follows demand allocation. It loads the weights, offloads the chosen layers to the GPU, and creates a KV cache only for the session that is open. When nothing is running, the memory footprint sits near the weight size plus a small engine buffer.
A serving engine follows reservation. To handle many requests without stalling, it keeps a pool of VRAM set aside for the scheduler and for KV cache blocks before any request arrives. That reserve is the reason the same model can look heavier at idle under a serving engine than under a desktop runner. Nothing is being wasted; the memory is held ready so concurrent requests do not have to wait for allocation.
This is the central point: the difference is reservation versus demand allocation, not a change in the model itself. A reader comparing task manager readings across two engines is seeing two memory philosophies, not two model sizes.
Matching an engine to the situation
The choice follows from how many people use the model and whether it sits on one desk or behind an endpoint.
For a single person on one machine, a single-user runner is the lean path. It keeps overhead low and lets the full card go to the model and its context. This is the usual fit for private chat, local coding help, or experimentation where one session runs at a time.
For a small team sharing one box, the same runner can work if usage never overlaps heavily, but the moment several people query at once, the single-session design becomes the bottleneck. The machine is not out of memory; it is out of concurrency.
For a service exposed to many users, a serving engine earns its reserve. Its paged cache and batching turn concurrent requests into shared, amortized work instead of separate full sessions. The higher baseline footprint is the price of that scaling, and it is paid back once real concurrency appears.
The local LLM hardware guide frames the same decision from the hardware side, and the Ollama VRAM requirements page shows how one runner maps common models onto real cards.
What to change first when VRAM runs out
When a model fails to load or an out-of-memory error appears, the order of fixes matters because the levers affect different parts of the bill.
The first question is whether the weights themselves fit. Offload settings, context size, and engine choice all sit on top of the weight size. If the weights alone exceed the card, no engine switch will rescue the load, because every engine must hold the same weights. Lowering quantization is the lever that actually shrinks the weights, and the quantization formats compared page shows how much each step saves. The GGUF vs safetensors page explains why one format is friendlier to partial offload.
The second question, once weights fit, is whether the overhead or concurrency is the problem. If a single-user runner fits the model but a serving engine does not, the reserve is the cause, and switching to a lighter runner or shrinking the reserved pool is the right move. If one session fits but many at once do not, the issue is KV cache growth under concurrency, addressed by context limits or a serving engine that reuses cache blocks rather than by changing quantization.
So the order is: confirm the weights fit by adjusting quantization, then manage overhead and concurrency through engine and settings. Switching engines is a refinement once the model loads, not the first response to a model that will not load at all.
Offload behavior across the runners
llama.cpp and Ollama share the layer-offload model: a setting chooses how many transformer layers run on the GPU, and the rest stay on CPU with system RAM. This per-layer control is what lets a large model run on a small card, at the cost of speed, as the llama.cpp offload layers page details. The memory used is the offloaded layers plus the KV cache, with nothing reserved for unseen sessions.
vLLM does not expose the same per-layer offload dial in the same way for a typical local setup. It assumes the model sits in accelerator memory and manages request memory through its paged cache. That design trades fine-grained layer control for smooth multi-request handling, which is the correct trade for a server and the wrong trade for a single desktop session.
The practical read: a reader who wants to squeeze a model onto limited VRAM should reach for the layer-offload runner, while a reader who wants many users served should reach for the serving engine and size its reserved pool deliberately.
KV cache handling in detail
The KV cache is where engines diverge most in memory character. In a single-user runner, the cache is created for the open context and released when the session ends. Memory tracks the one user’s context directly.
In a serving engine, the KV cache is split into blocks that are allocated, reused, and freed per request through a paging scheme. This avoids reserving the full worst-case context for every slot at once, which is what makes high concurrency feasible. The trade is the reserved management space and the scheduler state that must exist before any request shows up.
Both approaches store the same kind of data; the difference is whether cache memory is tied one-to-one to a live session or pooled and shared across many. For a reader planning capacity, the KV cache size explained page gives the formula behind the cache, and the context length cost page shows how context length feeds it.
Apple Silicon and unified memory
On Apple Silicon the boundary between VRAM and system memory is soft, because both draw from one pool. A single-user runner benefits here because offloaded and CPU-resident layers pull from the same unified memory, easing the fit for larger models. The Apple Silicon unified memory page covers the practical limits. A serving engine still reserves its pool, so the same reservation-versus-demand gap appears, just inside one memory space instead of a separate GPU card.
Quick decision checklist
- One person, one machine, privacy-first: single-user runner, offload as many layers as fit.
- One machine, light sharing, rare overlap: single-user runner, watch concurrent context.
- Several users, API endpoint, steady load: serving engine, plan for its reserved pool.
- Model will not load at all: shrink quantization before changing engine.
The multi-GPU VRAM page covers splitting a model across cards for the cases where one pool is not enough, and the measure VRAM usage page shows how to read the actual footprint once a runner is chosen.
What this page does not tell you
- This page gives no measured VRAM figures and no throughput numbers for any engine. Footprints shift with version, build flags, and settings, so any fixed number here would be wrong on another setup.
- This page does not cover version-to-version differences. An engine upgrade can change buffer sizes and reservation behavior, which is why readings should be re-checked after updates.
- This page does not cover deployment or operations cost: uptime, monitoring, scaling, security, and maintenance are separate concerns from memory shape and are out of scope here.