LOCAL AI · MEMORY PLANNING
Will it fit? Know before you download.
Estimate model weights, KV cache and runtime headroom. Compare your setup with the memory you actually have.
- No model download
- Runs on your device
- Transparent formulas
CALCULATORS
Run the numbers first.
- 01
LLM VRAM Calculator: Weights, KV Cache & GPU Fit
Estimate LLM GPU memory from weights, KV cache and runtime reserve. Compare GB and GiB, adjust context length and see the full calculation.
Open → - 02
KV Cache Calculator: Context Memory in GB and GiB
A working KV cache calculator: enter layers, KV heads, head dimension, context length and cache precision to estimate the cache in GB and GiB, including concurrent sequences.
Open → - 03
Context Length Calculator: Memory Budget in GiB
Enter a model's attention shape and available VRAM to see the longest context that fits after the weights, with a 4K/8K/32K/128K reference table and the exact breaking point.
Open → - 04
Token Cost Calculator: Local vs API Break-Even
Estimate whether running a model locally beats API tiers: local power and amortization vs budget, mid, and frontier API pricing, with a break-even month count.
Open →
BACKGROUND
Why the numbers look like that.
-
Run an LLM Locally: What You Actually Need
The decision path from picking a model to getting it running on your own GPU, with links to every VRAM and quantization tool on this site.
-
Model File Size Lookup: Common Model Sizes by Quantization
Common model weights across GGUF quant tiers, computed from bits per parameter, plus a reverse lookup from your VRAM budget to the model and tier that fit.
-
Model Catalog: File Sizes, Cloud Listings and VRAM Estimates
Compare published local model file sizes, context windows and estimates of weight memory. Cloud entries with no downloadable tag keep their size unknown.
-
KV Cache Size Explained: The Formula Behind Context Length
Why long context eats VRAM: the KV cache formula, how GQA shrinks it, and a worked table showing KV cache doubling when you double the context length.
-
GGUF Quantization Compared: What Q4_K_M Actually Costs You
Which GGUF quant tier to pick, and why: Q2_K to F16 compared on bits per parameter, 7B/13B/70B weight size, and the quality you trade away.
-
Can You Actually Run a 70B Model Locally?
A feasibility matrix for running a 70B LLM locally across GGUF quant tiers and common GPUs, including multi-GPU and CPU-offload outcomes.
-
Ollama Out of Memory: What Actually Frees VRAM
Ollama out-of-memory fixes that work: the real causes, what num_gpu and num_ctx do, and a symptom-to-root-cause table you can act on.
-
llama.cpp GPU Offload Layers (-ngl) Explained
What llama.cpp -ngl does: how many layers to offload to your GPU, the speed and memory tradeoff from -ngl 0 to all layers, and copy-paste command examples.
-
Apple Silicon Unified Memory for LLMs: How Much Do You Need?
How Apple Silicon unified memory fits LLMs: a tier-to-max-model table, why macOS reserves part of the pool, and how much you can really give a model.
-
GPU VRAM Reference: What Each Card Actually Gives You
A VRAM reference for local LLMs: nominal memory of common NVIDIA, AMD, and Apple cards, plus why usable VRAM is always less than the spec sheet says.
-
MoE Models and VRAM: Total vs Active Parameters
Why MoE VRAM depends on total parameters, not active ones, and how that differs from a dense model of the same size when fitting locally.
-
Context Length Costs VRAM: Sizing a Real Session
How context length drives KV cache VRAM: a chat-length to KV-cache table plus how batching and concurrency multiply memory on top of the model weights.
-
How to Measure Real VRAM Usage While Running a Model
Three commands to measure actual VRAM while a model runs on NVIDIA, macOS, and Linux, plus a table decoding what each reading means for your fit decision.
-
LLM VRAM Math Cheatsheet
Every VRAM formula for local LLMs on one page: weight size, KV cache, engine overhead, batch multipliers, and worked conversions you can do by hand.
-
Ollama VRAM Requirements: What Each Model Actually Loads
Estimate Ollama VRAM per model and quant tier, including weights, a 4K KV cache, and headroom — formula-based figures, not measured readings.
-
Fine-Tuning VRAM: LoRA, QLoRA and Full Weights
Why fine-tuning needs more VRAM than inference, the multiplier each method adds over inference, and what model scale a 24/48/80 GB card can hold for training.
-
llama.cpp vs Ollama vs vLLM: Which Runner Costs You Memory
A qualitative comparison of how llama.cpp, Ollama, and vLLM differ in VRAM overhead, startup, concurrency, and KV cache handling, so a reader can pick the right runner.
-
Which GPU for a Local LLM: A Sizing Guide
A capacity-only guide to picking local LLM hardware: VRAM brackets mapped to model scale per quant tier, plus the bandwidth and format specs that decide real-world fit.
-
LLM API Cost Comparison: How to Read Vendor Pricing
A pricing-dimension comparison table and a method to build your own LLM API cost tracker, instead of copying an expired price snapshot that drifts within weeks.
-
What a Chat AI Subscription Costs Per Month, and When Local Beats It
How monthly chat AI subscription tiers map to cost ranges and usage intensity, why prices differ by region, and at what usage a local model becomes the cheaper path.
-
Stable Diffusion VRAM: What Actually Occupies the Card
How image diffusion models spend VRAM: a breakdown of weights, encoders, VAE, resolution-driven activation, batching and ControlNet, and why resolution scales quadratically.
-
Whisper and Audio Models: Sizing VRAM for Speech
How speech and audio models spend VRAM: encoder and decoder weights, chunk duration, batching, overlap, and streaming residency, and why duration is the real context knob.
-
Vision Models and Multimodal LLMs: Where the Memory Goes
How vision encoders and multimodal LLMs spend VRAM: visual tokens from image patches, the projection layer, and why image cost ultimately lands on the KV cache.
-
Multi-GPU VRAM: How the Model Actually Gets Split
How a model's memory is divided across several GPUs: tensor parallel, pipeline parallel and layer offload, what each costs, and why two cards' VRAM does not simply add up.
-
CPU Offload: How Much VRAM You Save, and What It Costs
CPU offload trades VRAM for speed. This page shows the tradeoff by offload ratio and why the slowdown is not linear, plus the order to try other levers first.
-
Embeddings and RAG: Memory Beyond the Chat Model
A RAG system holds more than the chat model: embedding weights, a reranker, and a vector index. This page breaks down that separate memory budget and how to size it.
-
Batch Size and VRAM: Why Serving Costs More Than Chatting
Why serving a model to many users needs more VRAM than chatting alone: the KV cache multiplies per concurrent session, on top of the shared model weights.
-
GGUF vs Safetensors: What Changes on Disk and in Memory
GGUF and Safetensors are not the same kind of file. This page compares disk size, load-time VRAM behavior, quantizability, and which toolchains each suits.
-
Quantization Quality Loss: What Is Known, What Is Assumed
A source-graded breakdown of what quantization does to model behavior: directional claims on quality loss, evidence strength, and how to verify it yourself.
-
AI and Spreadsheets: Where a Local Model Fits and Where It Does Not
A task-by-task look at when a local model helps with spreadsheet work and when a cloud API is the better call, based on data residency and task volume.
Pages on this site
How these numbers stay current
No marketing copy on this site. Every page answers one question about running large models on your own hardware: how much memory it needs, why it needs that much, and how to check before you download forty gigabytes of weights.
Model sizes, quantization formats and runtime overhead change often. When a figure shifts, the existing page is corrected in place — the URL stays the same, the number underneath it is what gets fixed. The arithmetic behind each result is shown on the page, so you can disagree with it.