The baseline: Llama 3.3 70B at Q4_K_M

About 40.7 GB of weights. Add a KV cache and an activation buffer:

Option 1: two RTX 3090s (48 GB total)

• 4K: 46.1 of 48 GB, 96% of the pair. It loads, but there is no margin, so plan for a headless box.
• llama.cpp's auto-fit keeps ~1 GiB free per card, so its effective budget is ~46 GB. Pass -ngl all and an explicit -c so a model that doesn't fit fails loudly instead of silently offloading layers to CPU.
• No NVLink needed: in the default layer split only boundary activations cross cards.
• Want comfort? A generic Q3_K_M of a 70B at 8K is ~38.4 GB.

Option 2: DGX Spark 64GB

• Llama 3.3 70B Q4_K_M at 32K: ~55.7 GB, inside the comfortable line. This is "the dense-70B machine".
• GPT-OSS 120B does not fit: MXFP4 at 8K is ~66.5 GB, Q4_K_M at 8K is ~62.9 GB (98% of the pool, with the OS sharing it).

Option 3: DGX Spark 128GB

• Everything above, plus the 100B-class MoE releases: GPT-OSS 120B, GLM-4.5-Air Q4_K_M (~68.7 GB at 8K), Llama 4 Scout Q4_K_M (~70.7 GB at 8K).
• At 4K context, 116 of the 129 models in the index fit comfortably (111 on the 64GB, 103 on a 32GB RTX 5090).

Speed is a different story

Both Sparks have 273 GB/s. Generation reads the whole dense model per token, so Llama 3.3 70B Q4_K_M tops out around 7 tok/s on either. An RTX 5090 moves 1,792 GB/s, about 6.6x more, but only on models it can hold. MoE models read only active experts, which is why they feel much faster on a Spark.

Rule of thumb

• Dense 70B, short context, budget: dual 24GB cards.
• Dense 70B with long context: Spark 64GB.
• 100B+ MoE: Spark 128GB.

Full walkthroughs with commands:

• Dual 3090 + llama.cpp: https://quantized.uk/cookbook/dual-gpu-70b-llamacpp/
• DGX Spark 64GB vs 128GB: https://quantized.uk/cookbook/dgx-spark-64gb-vs-128gb/
• Try your own model/context: https://quantized.uk/tools/vram-calc/

All numbers are calculator estimates; if your measured usage differs, I'd like to hear it.

1. Memory = weights + KV cache + a little buffer

The GGUF file size is not your VRAM requirement. On top of the weights you need the KV cache, which grows with context length, plus some activation buffer. That is why the same model can fit at 4K context and fail at 32K.

2. Worked example: Qwen3.8 27B (GGUF)

So on a 24GB card (RTX 3090/4090, RX 7900 XTX), Q4_K_M leaves ~5.5 GB for longer context; Q5_K_M fits but is tight. On a 16GB card, look at Q3_K_M or partial CPU offload. Qwen3.8 uses a hybrid layout where only some layers keep a KV cache, so its long-context cost is far lower than a conventional 27B.

3. 24GB vs 32GB: what the RTX 5090 really buys

• At 4K context, 97 of the 129 models in the quantized.uk index fit a 24GB card comfortably (102 load at all). On the 32GB RTX 5090 it's 103.
• A 70B at Q4_K_M with 8K context is about 47.5 GB, so one 5090 does not change that: you still need two cards or offloading.
• What the extra 8GB buys is context room on 27B–36B models, plus bandwidth: 1,792 GB/s vs 1,008 GB/s on the 4090.

4. Bandwidth decides speed, VRAM decides fit

Cards with the same 24GB run the same models at very different speeds: memory bandwidth ranges from 346 GB/s (Tesla P40) to 1,008 GB/s (RTX 4090). Generation speed roughly scales with bandwidth ÷ weight file size.

5. Macs

A Mac's GPU can only use part of unified memory (about three quarters on a 24GB Mac), so size against that, not the full number.

Tools

I maintain these free, no-signup references:

• VRAM calculator (model + quant + context → weights + KV cache): https://quantized.uk/tools/vram-calc/
• Command generator for llama.cpp / Ollama / vLLM: https://quantized.uk/tools/cli-gen/
• What to run on 8GB / 24GB / RTX 5090: https://quantized.uk/cookbook/

Numbers are estimates plus community measurements; if your real usage differs, tell me and I'll fix it.