Speech and audio models look, at a glance, like language models: they have an encoder, they produce text, and they run on the same hardware. Their memory behavior is different in one important respect. A text model’s working memory grows with how many tokens are in context, an amount that can be set very large. An audio model’s working memory grows with how much sound is in the current chunk, and that chunk is usually a fixed, short window. This page explains where the VRAM goes in an audio pipeline, why audio models tend to be easier to fit than text models of comparable parameter count, and why chunk duration is the lever that matters most.
The weight portion: same arithmetic as everywhere else
As with any neural network, the model weights are the baseline cost. The encoder that turns audio features into representations and the decoder that turns those representations into text tokens each carry weights that must be resident. The size of those weights follows the standard rule: parameter count times bits per parameter, divided by eight, as the VRAM math cheatsheet lays out.
A smaller audio model fits easily on modest hardware; a larger one needs more room, exactly as expected. The weight floor is set by the quant, and the quantization formats compared page describes how bits per parameter trade off against fidelity. The point to keep here is that the weights are a fixed baseline, paid once, and shared across every chunk of audio.
The audio context is duration, not token count
The defining difference from a text model is what counts as “context.” In a language model, context is a sequence of tokens that can be set to thousands or hundreds of thousands of positions, and the KV cache scales linearly with that length. In an audio model, the input is a waveform or a spectrogram, and it is almost never fed in all at once. Instead it is divided into chunks of a few seconds to a few tens of seconds.
Because each chunk is bounded, the amount of audio the encoder must hold at one time is bounded. The model does not accumulate an ever-growing cache for the entire recording the way a text model accumulates cache for an entire conversation. That bounded window is why an audio model’s peak memory stays predictable even for very long source material.
This is also why the “context” analogy breaks down. A text model’s context is an open dial; an audio model’s context is a fixed-size sliding window. The memory therefore does not drift upward with total duration, only with chunk size.
Chunking is the real VRAM switch
The single most important setting for an audio model’s memory is the chunk or segment length. Longer chunks mean a larger feature map entering the encoder, which means more activation memory per step. Shorter chunks keep that map small.
The relationship is roughly linear in chunk duration for a fixed model: doubling the chunk length approximately doubles the audio frames the encoder processes, and the activation grows in step. This is a gentler law than the quadratic resolution growth of image models, and a different law from the token-linear growth of text models, but it is still a direct multiplier on the working set.
A practical rule: if an audio pipeline overflows, shortening the chunk is the most reliable way to bring the peak down. Lengthening the chunk to capture more context does cost memory, and that cost is paid on every chunk, so it accumulates across the whole job in throughput terms even though peak stays at the per-chunk level.
The audio-task VRAM breakdown
The table separates the memory components of a typical speech or audio pipeline. As with the image page, the figures are qualitative: exact bytes depend on the model family, parameter count, quant, chunk length, and engine. The value is in seeing which rows move and which stay fixed.
| Component | What it is | Scales with | Often missed? |
|---|---|---|---|
| Encoder weights | Audio feature extractor | Model size and quant bits | No |
| Decoder weights | Text token generator | Model size and quant bits | No |
| Per-chunk activation | Feature map for the current window | Chunk duration (linear) | Yes — set by chunking, not duration |
| Batch of chunks | Several chunks at once | Roughly linearly with count | Partly |
| Overlap margin | Boundary audio kept between chunks | Overlap fraction of chunk | Yes — small but real |
| Streaming residency | Model kept loaded per live stream | Number of concurrent streams | Yes — steady overhead in service |
| Long-audio buffer | Reassembly of many chunks | Disk/CPU, not VRAM | Mostly no |
The rows marked “often missed” explain why two setups with the “same model” show different memory: one may use long chunks, overlap, and a resident streaming service, while the other processes short batches offline. The weights are identical; the working set is not.
Long audio is handled by splitting, not by growing memory
A common worry is that a two-hour meeting needs twice the VRAM of a one-hour meeting. Under chunked processing it does not. The recording is cut into fixed windows, each window is transcribed within the chunk budget, and the results are stitched together. Total duration changes how long the job takes and how much disk the intermediate features use, but the peak VRAM is set by the chunk size, not by the recording length.
This is the opposite of a text model asked for a 128k context, where enabling the long context reserves a large cache whether or not it is fully used. An audio model naturally bounds its working memory by construction, which is a large part of why audio models are considered friendly to fit.
Overlap and why it is cheap but not free
Chunk boundaries are a known weak spot: a word split across the cut may be misread. A common fix is to keep a small overlap, a margin of audio carried from the end of one chunk into the start of the next. Overlap enlarges each chunk slightly, so it adds a little activation, but the dominant cost remains the base chunk length. A modest overlap is usually a small price for cleaner boundaries and rarely changes the fit decision.
Worth noting: overlap affects per-chunk size, not total duration, so its memory effect is bounded and predictable. It is the kind of detail that shows up only when comparing two runs that differ in boundary handling.
Batching and concurrency
Running several chunks in a batch multiplies the per-chunk activation, just as batching multiplies activations in image models and multiplies KV caches in text serving. A batch of N chunks at a fixed length raises activation by roughly a factor of N on top of the shared weights. Serving multiple audio streams at once adds a steady resident overhead, because each live stream keeps the model loaded and holds its own position state.
The weight portion is shared, which is why a single model can serve many short jobs; the variable part is the per-chunk working set and, in a service, the number of concurrent streams. Lowering the batch or the concurrency is the usual remedy when an audio pipeline runs out of memory, and it works for the same reason it works elsewhere: the weights are not the problem, the working set is.
Real-time streaming versus offline batch
Offline transcription loads the model, processes a file in chunks, and unloads. Streaming transcription keeps the model resident for the life of the connection and maintains state per stream. The weights are the same in both cases, but the streaming setup carries a more constant resident overhead and must hold memory steady under load rather than peaking once and releasing.
Peak per-chunk memory is set by chunk size either way, so the sizing question is the same: pick a chunk length the card can hold, then decide how many streams to allow. The local LLM hardware guide gives judgment criteria for how much memory a workload needs, and those criteria apply to audio services as well. The GPU VRAM reference lists nominal capacities, with the usual caveat that usable memory is below nominal.
Why audio models fit more easily than text models of similar size
Putting the pieces together: an audio model of a given parameter count tends to be easier to host than a text model of the same count because its working memory is bounded by a short audio window rather than by an open token context. The text model can be asked for a huge context that reserves a large cache; the audio model cannot be asked for “more context” in the same way, because its context is the chunk, and the chunk is fixed by the pipeline.
That is not a statement about quality or speed, only about the memory envelope. The weights dominate, the activation is small for typical chunks, and the total duration does not raise the peak. For someone choosing what to run locally, an audio model is often the gentler introduction to a model family of that size.
Connecting audio models to the rest of the site
The weight term is the same arithmetic used across the site: see the model file size lookup for how parameter count and quant produce a file and weight size. The context length cost page is useful as a contrast: it shows the linear token growth that audio models avoid by chunking. The batch size and throughput page explains the linear multiplier that batching applies to any working set, audio included. And the measure VRAM usage page describes how to observe the real peak rather than guess it, which is the right move before trusting any chunk-length choice.
What this page does not tell you
This page does not give a per-model VRAM number for any specific speech or audio model, because the figures depend on the architecture, quant, chunk length, overlap, batch, and engine, and a single number would mislead at other settings. It does not cover training or fine-tuning, which layer gradients and optimizer state on top of the weights described here. It does not benchmark transcription speed, which depends on the runtime and hardware rather than on the memory layout. It does not recommend specific graphics cards; the GPU VRAM reference gives nominal capacities only, and usable memory is always less. It also does not address music separation, speaker diarization, or other audio tasks whose memory patterns differ from straight transcription.