Loading page
About this tool
Compare LLM file sizes across FP16, 8-bit, 6-bit, 5-bit, 4-bit, 3-bit and 2-bit quantization formats.
The Quantization Size Calculator estimates a language model's weight-file size with the formula size = parameters × bits-per-weight ÷ 8, across FP32, FP16, Q8_0, Q6_K, Q5_K, Q4_K, Q3_K and Q2_K. Bits-per-weight figures come from the exact llama.cpp GGUF block layouts — for example Q8_0 stores 34 bytes per 32 weights, which is 8.5 bits per weight. It is built for people running local models with Ollama, llama.cpp or LM Studio who need to know whether a given quant will fit their RAM or VRAM before downloading it.
Open Quantization Size Calculator on AltFTool — it loads instantly in your browser.
Enter the values you already know.
Fine-tune the options to match your scenario.
Read the result and use it in your planning or reporting.
GGUF figures are derived from the real llama.cpp block structures — Q8_0 is 8.5, Q6_K is 6.5625, Q4_K is 4.5 bits per weight — not rounded guesses.
See both decimal gigabytes (download-page style) and binary gibibytes (what your OS reports).
Every format shows its percentage saving against half precision, the usual distribution format.
Multiply the parameter count by the bits per weight and divide by 8. A 7B model at FP16 (16 bits/weight) is 7 × 10⁹ × 16 ÷ 8 = 14 GB of weights; the same model at GGUF Q4_K (4.5 bits/weight) is about 3.9 GB. Real files run a few percent larger because of metadata and tensors kept at higher precision.
About 39 GB of weights: 70 × 10⁹ parameters × 4.5 bits/weight ÷ 8 ≈ 39.4 GB (36.7 GiB). At FP16 the same model needs about 140 GB, so 4-bit quantization cuts the footprint by roughly 72%. You still need additional memory for the KV cache when running it.
Because GGUF Q8_0 stores weights in blocks of 32 with a 2-byte FP16 scale per block: 32 int8 values plus 2 scale bytes is 34 bytes per 32 weights, which works out to exactly 8.5 bits per weight. Every GGUF format carries similar per-block scale overhead, which is why Q4_K is 4.5 bits rather than a flat 4.
No — the file size is the floor, not the total. Inference also needs the KV cache, which grows with context length and can add several gigabytes at long contexts, plus compute buffers and your OS overhead. A practical rule is to leave at least 1–2 GB of headroom beyond the file size for small contexts, and more for long ones.
Add the Quantization Size Calculatorwidget to your blog or website — free, responsive, no signup. Just keep the “Widget by AltFTool” credit link visible.
<iframe src="https://www.altftool.com/embed/widget/quantization-size-calculator"
title="Quantization Size Calculator — free AltFTool widget"
width="100%" height="640" style="border:0;border-radius:12px;overflow:hidden"
loading="lazy" referrerpolicy="no-referrer-when-downgrade"></iframe>
<p style="font-size:12px;margin:4px 0 0">Widget by <a href="https://www.altftool.com/tools/all/quantization-size-calculator?utm_source=embed&utm_medium=widget">AltFTool — free online tools</a></p>