Fine-Tuning VRAM: LoRA, QLoRA and Full Weights

Why fine-tuning needs more VRAM than inference, the multiplier each method adds over inference, and what model scale a 24/48/80 GB card can hold for training.

By VRAM Fit editorial Published 2026-09-17 Updated 2026-09-17

Fine-tuning a model almost always needs more VRAM than running it. The gap is not a rounding error: the same weights that fit comfortably for inference can require several times the memory to train, because training keeps extra states resident that inference never touches. This page explains why the gap exists, gives the multiplier each common method adds over inference, and shows what model scale a 24 GB, 48 GB, or 80 GB card can realistically hold for training.

Why fine-tuning eats more VRAM than inference

Inference holds the model weights and a KV cache. Training holds those and three more things, each large on its own:

Weights. The model parameters stay resident for the forward and backward pass, exactly as in inference.
Gradients. Backpropagation computes one gradient per trainable parameter. At full precision that is another full copy of the parameter count, so the gradients alone can equal the weight size.
Optimizer states. An optimizer such as Adam keeps running statistics per parameter, often two copies at the optimizer’s precision. This is the largest single extra block and the main reason full fine-tuning is so heavy.
Activations. The forward pass intermediate tensors must be stored to compute gradients, and they scale with batch size and sequence length rather than with the optimizer choice.

The weights and gradients are roughly fixed multiples of the parameter count. The optimizer states are the variable that different methods attack hardest, and activations are the term that long context pushes up.

Method to VRAM multiple, over inference

The table below gives the order-of-magnitude multiplier of each method relative to the same model run for inference. These are ranges, not fixed numbers. The actual value moves with the optimizer, the precision, whether gradient checkpointing is on, and whether any state is offloaded to CPU.

Method	Multiple over inference (estimate)	What drives it
Full fine-tune, FP16 + Adam	~4 to 6 times	Weights + gradients + two FP16 optimizer states + activations
Full fine-tune, BF16 + 8-bit optimizer	~2 to 3 times	Optimizer states are quantized, the single biggest saving
LoRA, BF16 base + adapter	~1.2 to 1.5 times	Base weights still resident; the adapter added is small
QLoRA, 4-bit base + LoRA	~0.7 to 1.0 times	4-bit base already resident; needs a paged optimizer on top
Doubling sequence length or batch	Activations grow roughly linearly	Activations are the residual term, independent of the optimizer

The inference baseline used here is the model’s resident weight footprint: BF16 for full and LoRA training, and a 4-bit (Q4_K_M-class) footprint for QLoRA. The multiple is a planning estimate, not a measured value, and should be treated as a capacity bracket rather than a point figure.

VRAM bracket to trainable model scale

The matrix below pairs a single-card VRAM bracket with the model scale each method can hold as a resident training load. The weight baselines are taken from the site’s GGUF size table: a 7B model is about 3.9 GB at Q4_K_M and 14.0 GB at BF16, a 13B model about 7.3 GB at Q4_K_M and 26.0 GB at BF16, and a 70B model about 39.4 GB at Q4_K_M and 140.0 GB at BF16.

VRAM bracket	Full fine-tune (FP16+Adam)	Full fine-tune (BF16+8-bit)	LoRA (BF16 base)	QLoRA (4-bit base)
24 GB	Below 7B; the 14 GB BF16 base alone leaves little for gradients and optimizer	Below 7B; 7B at 2x is ~28 GB, over budget	7B-class (16.8 to 21 GB)	Up to ~13B at 4-bit; 70B base alone at 39 GB is out of range
48 GB	7B at the low end is marginal; most loads need more	7B comfortable; 13B at the low end is marginal	7B and 13B (31 to 39 GB)	Holds the 4-bit base of a 70B-class model with paged optimizer room
80 GB	7B comfortable; 13B at the low end is marginal	7B and 13B (52 to 78 GB)	7B, 13B, and mid-scale models up to ~34B at the low end	70B at 4-bit with substantial headroom for context and optimizer

This matrix is a capacity judgment, not a count of how many cards a training job requires. The bracket width is set by optimizer precision and activation size, both of which shift with configuration. The GPU VRAM reference lists nominal card sizes to compare against these brackets, and the model file size lookup gives the exact weight figures behind each row.

Three levers that shrink training memory

When a model does not fit, three independent moves reduce the resident load, and they stack:

Quantize the base. Training on a 4-bit base instead of BF16 is the difference between the QLoRA and LoRA rows above. The base shrinks by roughly 3.5x, and only the small adapter is kept at higher precision. The tradeoff is training quality, which moves with the model and task rather than with a single number.
Use a paged optimizer. Offloading optimizer state to CPU or to paged memory removes the largest fixed block. It lowers the resident footprint at the cost of transfer time, which is the subject of the CPU offload performance page.
Checkpoint gradients and shorten the sequence. Gradient checkpointing trades compute for activation memory, and cutting batch size or sequence length shrinks activations roughly linearly. Because activations scale with context, the context length cost page is the relevant budget to watch for long-document training.

Training and inference budgets do not swap

A model that needs 80 GB to train may serve in far less once gradients and optimizer states are freed. The reverse is also true: a model that serves at 24 GB will not train there without one of the levers above. The two budgets describe different resident states and should be sized separately. The VRAM calculator is built around the inference shape; training needs the extra terms added explicitly before any card is ruled in or out.

What this page does not tell you

It does not list memory figures for specific training frameworks. Optimizer layout and checkpointing behavior differ between implementations, and this page gives only the structural terms.
It does not give throughput or wall-clock time. Speed depends on bandwidth, driver, and offload ratio, none of which this page estimates.
It does not model dataset size. The number of training samples changes steps and convergence, not the resident memory bracket described here.

For actual resident usage on a given rig, measure the process directly with the measure VRAM usage guide rather than trusting a planning multiple.

Asked before trying it

Why does fine-tuning use more VRAM than just running the model?

Running a model needs only the weights and the KV cache. Fine-tuning additionally keeps gradients, optimizer states, and activations resident, so the same model can need several times more memory to train than to serve.

How much more VRAM does full fine-tuning need?

As an estimate, full fine-tuning in FP16 with Adam needs roughly 4 to 6 times the inference memory, and BF16 with an 8-bit optimizer needs roughly 2 to 3 times. The exact figure moves with the optimizer, precision, checkpointing, and any offload used.

Is QLoRA really cheaper than inference?

On a 4-bit base it can land near or below the inference footprint, about 0.7 to 1.0 times, because the resident base is already 4-bit and only the small adapter plus a paged optimizer is added. The range depends on sequence length and batch size.

Can a 24 GB card fine-tune a 70B model?

Not by the resident-weight estimate alone: a 70B model at Q4_K_M is about 39 GB, which already exceeds 24 GB. QLoRA with heavy CPU offload of optimizer state changes the picture, but capacity planning should start from the 39 GB base figure.

Does a bigger batch size change the training multiplier?

It changes the activation term, not the weight, gradient, or optimizer terms. Doubling batch size or sequence length grows activations roughly linearly, which is why long-context training is often the real memory ceiling.

Can the training memory estimate be reused for serving the model?

No. The training budget and the serving budget are different shapes. A model that needs 80 GB to train may serve at a fraction of that once gradients and optimizer states are released, so the two budgets must not be swapped.