Best for your hardware
Quality vs speed map · every model at its recommended quant
All models
How we estimate · formulas, sources and confidence
All numbers are estimates for llama.cpp-based runtimes (llama.cpp, LM Studio, Ollama) at batch size 1, Flash Attention on. They are computed in your browser by engine.js; nothing is sent anywhere.
Memory: does it fit?
- Weights = the real GGUF file size from Hugging Face (all split parts summed). Without a file the size is estimated from bits per weight with a size-class correction.
- KV cache per layer group: full-attention layers grow with context (cells padded to 256); sliding-window layers hold only window + 512 cells; MLA stores one latent vector; hybrid (DeltaNet/Mamba) layers keep a constant recurrent state. Bytes per element: f16 2.0, q8_0 1.0625, q4_0 0.5625.
- Usable VRAM = VRAM × GPUs − desktop reserve (Windows 0.8 GiB, Linux 0.4 GiB) − 1 GiB margin per GPU (llama.cpp
--fitdefault), minus a compute buffer and backend overhead. Apple: ~67% (≤36 GB) or ~75% of unified memory. Memory is shown in GiB (2^30 bytes, like VRAM and RAM sizes); file sizes in GB as on Hugging Face. - Compute buffer = ubatch × vocabulary × 4 bytes + activations. Offloaded configs for long prompts (coding, long context) use
-ub 2048, which streams offloaded weights over PCIe 4× less often during prefill; the larger buffer is included in the fit and the command uses the same value. - Verdicts: ✓Fully on GPU ≤92% of usable · ◐Tight ≤100% · ⇄MoE: experts in RAM dense parts on GPU, some expert layers via
--n-cpu-moe· ◑Partial offload whole layers on the CPU · ✕Too large exceeds VRAM + RAM.
Speed
- Generation is memory-bandwidth bound:
t/token = t0 + bytes_on_GPU / (BW × efficiency × quant factor) + bytes_in_RAM / (RAM BW × CPU efficiency), including the KV cache read at a typical depth for the use case: a multi-turn session (chat, coding, reasoning, vision) sits at half the window on average (at least the prompt plus half a typical answer), long context at the document (70% of the window) plus half the answer. The details also show the speed of the first replies and of a completely full window. Efficiencies are calibrated against llama.cpp scoreboard results (Llama-2-7B Q4_0). - MoE models only read their active experts per token. Offloaded experts are read from system RAM, which is why RAM type and channel count matter.
- Prefill (time to first token) scales measured pp512 results by active parameters, quant type and prompt length (chat 1k, coding 8k, reasoning 2k, vision 1.5k tokens; long context 70% of the window, so the wait grows with the window; every prompt leaves room for a typical answer). Prefill slows with depth: at a 32k depth a dense model keeps about 67% of its short-prompt rate, a full-attention MoE about 45% (Qwen3-30B-A3B measured), an MLA model about 25%. Only layers that attend over the whole window cause the extra MoE/MLA drop, so it scales with their share: gpt-oss (half its layers on a short sliding window) keeps about 62% (measured 60–64%), hybrids with mostly linear-attention or sliding-window layers (Qwen3.5/3.6-35B-A3B, Qwen3.8-Flash-Next) fall off like a dense model. The details show this effective rate over the prompt next to the short-prompt rate. With an offload only the GPU part of the prefill slows with depth; streaming the offloaded weights over PCIe does not.
- One speed: the tok/s shown everywhere (hero, list, map, quant ladder, why texts) is the expected speed at that typical depth including the placement deduction: an offload counts as 25% slower, a tight fit on Windows + NVIDIA as 10% slower (a VRAM overflow silently spills into system memory). The minimum speed, the usability labels and “below target” use exactly this number, and so does the ranking for chat, reasoning and vision. The time to first token is a wait, not a speed: it is shown on its own; for coding and long context the ranking discounts this number by the wait on top (√(target ÷ wait) when the wait is over its target: coding 4 s, long context 8 s per 16k prompt tokens, i.e. about 45 s for the ~90k-token prompt of a 128k window), and comparisons name the wait when it is what decides. ⚡ “Quickest answer” marks a faster alternative whose tok/s match ★ and whose edge is the shorter wait.
- Ranges are the model’s own error around the shown value: ±12% for a calibrated GPU with everything on it, ±20% with a default efficiency, ±40% with offloading (MoE fully on an AMD GPU: −45%, one measurement shows slower routing there). With an offload the upper end still holds the estimate before the 25% deduction. The 10% tight-fit deduction on Windows + NVIDIA stands for the risk of a spill into system memory (5–10× slower if it happens), not for a typical loss, so no range covers that case; the warning under “Good to know” names it. Calibrated means a measured GPU efficiency and a real file size; Estimated means at least one input is a default.
Quality and the quant pick
- Quality retained is derived from the measured KL divergence of each quant type vs. full precision, scaled by model size (large models tolerate low bits better; MoE uses √(total × active)) and by use case (coding and reasoning are more sensitive). Quantized KV cache costs a little extra.
- Accuracy: the retention curve matches measured benchmark losses only to about ±2 points, and benchmark differences under ~0.5 point are noise, so quality is shown in ½-point steps, rounded down (so a shown value never lands on the wrong side of the 85% floor or the quality knee; only a lossless file shows 100%), with an estimated range (±40% of the estimated loss, at least ±0.5 and at most ±2 points: wider for lower-bit quants and smaller models), and values at most one ½-point step apart read “same quality, within noise”. This rounding is display only: the pick still uses the unrounded value with the 0.2-point noise rule below. The benchmark score of the model itself (llm-stats or a curated estimate) is the larger uncertainty.
- The recommended quant maximises
ln(quality) + λ · ln(min(speed, cap) / cap), where the cap is twice the use-case target speed and λ comes from the Speed ↔ Quality slider; towards “Speed”, speed between 2× and 4× the target still counts at half weight. For coding and long context a slow time to first token lowers the effective speed. An offload costs ~1.5% quality-equivalent on top of its 25% speed deduction (prefill, RAM, uncertainty); a tight fit costs a little (more on Windows + NVIDIA, where an overflow silently spills into system memory). Above the cap (beyond the half-weight band towards “Speed”) speed no longer counts, except against an offload: an offload also pays for the speed it gives up above the cap against the fastest fully-on-GPU quant of the same model above the quality knee, with weight λ × (1 − slider position), i.e. the full below-cap weight at “Speed”, half of it at “Balanced” and none at “Quality”. At “Balanced” an offload next to a good GPU fit that is at least 2× faster would need about 4.3–5 points more quality retained (4.3 against a tight fit on Windows + NVIDIA), more than the knee usually leaves (it sits at 95–97%), so in practice the GPU fit wins there and towards “Speed”; towards “Quality” the slower, better offload can still win (Qwen3-Omni 30B on an RTX 4090: Q4_K_M on the GPU at ~161 tok/s up to 0.75, Q8_0 with experts in RAM at ~33 tok/s from 0.8; the why text then names the faster GPU fit). A quantized KV cache is only considered when f16 KV does not fit comfortably. - Hard rules: never below 85% quality retained (and IQ1 only for models over 200B); if a config above the quality knee (97% for models under ~10B, 95% from ~16B) reaches the minimum speed, the pick is one of those (unless the only such config is set aside by the bit-width rule below); never F16/BF16 over a near-lossless quant of at most Q8_0 size; a dense model of 20B+ is not squeezed below ~3.6 bits per weight (60B+: ~2.7) when a quant with more bits keeps at least the same quality, either without a riskier fit or because the low-bit one is below the quality knee anyway (such rows show “Too few bits” and are never ★, ◆ or ⚡); quality differences under 0.2 points count as noise and the faster option wins.
- If nothing above the knee reaches the minimum speed, the pick trades the two continuously. At “Balanced”, falling 10% short of the minimum weighs about as much as being 0.5% below the knee and running at half the minimum about as much as ~3.5% below it; towards “Speed” a squeeze below the knee gets cheaper (4× at the end), towards “Quality” dearer, so the slider also decides how far a model is squeezed. Below the minimum, speed counts much more than the slider alone would give it. Raising the minimum speed never makes the pick slower than a previous pick that already met the new minimum. While no good quant reaches the minimum, raising it can still shift the pick slightly in either direction; for coding and long context the trade uses effective speed (including the time to first token), so the generation speed shown can dip when a quant with faster prompt processing wins.
- Markers in the quant ladder: ★ recommended · ◆ max quality whose fit is no riskier than ★’s (no offload when ★ is fully on the GPU) that keeps at least half of ★’s speed (or still reaches the use-case target) and the minimum speed (or ★’s speed if ★ is slower); if nothing qualifies, ★ already is the max quality · ⚡ fastest that stays above the quality knee, or at ★’s quality if ★ is below it (never slower than ★).
Ranking models
- Model score = benchmark quality (weighted for the use case, 0–100) × quality retained by its recommended quant. The benchmark categories sit on different scales (code, tool calling, vision and long context run 14–25 points below the overall index, GPQA-based knowledge ~20 above it and wider), so each category is first mapped onto the overall scale (median offset over the catalog; knowledge also compressed to the overall spread). A missing category counts as the model’s overall score, so a model is neither rewarded nor punished for a benchmark it lacks. On the shared scale, score ratios between models are smaller than on the raw category scales (tool calling ran 10–40), so speed weighs a little more against score than before.
- Models are ranked by that score plus the same speed term (including the pick’s offload cost against its own GPU fit), with an extra penalty for falling short of the use-case target speed (chat 20, coding 30, reasoning 40 tok/s) whose weight never drops below its “Balanced” value, for a pick below the quality knee, and for a dense ≥20B model below ~3.6 bits per weight (≥60B: below ~2.7). Rows tagged “below target” or “heavy quant” explain why the rank differs from the score. Models without benchmark data come after scored ones.
- Benchmarks come from llm-stats.com (normalised) or a small curated table of hand estimates; they are self-reported by vendors and vary in coverage. The hero and the list tag the source (“curated” estimate, “no score”); the hero also names the runner-up and why the top pick beats it (for a close score also the quality each quant keeps and which score is only a curated estimate).
Known limits
- Not modelled: speculative decoding / MTP speed-ups, tensor-parallel split modes, longer thinking traces of low-bit reasoning models, and exact CPU prefill.
- MoE expert-offload speed can be optimistic; Apple M1–M4 numbers are fitted (with their ~5 ms per-token overhead) to older llama.cpp builds and are probably conservative for current builds.
- Your runtime's own estimator (e.g. LM Studio's guardrails) may disagree; it is known to over-estimate memory.
Copy commands
- Minimum build: where a documented minimum exists, the command starts with a comment line and “Good to know” warns: Qwen3.8-Flash-Next needs llama.cpp b10660 or newer, the Q2_0 quant type b11556 or newer (older builds silently produce garbage). LM Studio and Ollama need a llama.cpp-based runtime at that level; Ollama shows no llama.cpp build number, so if unsure use llama.cpp at that build or newer.
- Ollama on Windows: the Ollama app already runs the server in the taskbar on port 11434, so a second
ollama servefails, and it only reads its settings when it starts. Following Ollama's FAQ, the command says to quit Ollama from the taskbar and saves the settings as user variables (setx) for later starts from the Start menu.setxdoes not change the open PowerShell window, andollama runstarts the app from that window when no server answers, so each variable is also set there ($env:): the block works when pasted as one piece. The user variables then apply to every model (the context length too); a comment line gives the PowerShell command that removes them ([Environment]::SetEnvironmentVariable(name, [NullString]::Value, "User"); a real null deletes the variable, whereas PowerShell would pass$nullas an empty string, which PowerShell 7.5+ stores as an empty variable). With a partial offload the Ollama tab notes that Ollama chooses its own GPU/CPU layer split, so its speed can differ from the one shown (estimated for the llama.cpp-nglsplit). - Split GGUF files (
-00001-of-0000N):llama-server -hfdownloads every part and loads them from the first. Ollama cannot pull split files from Hugging Face (open issue ollama/ollama#5245, as of 2026-11), so its command downloads the parts (Hugging Face CLI) and imports them with a Modelfile (FROM ./…-*.gguf, as Ollama's import guide documents). LM Studio's docs do not cover split files: keep all parts in one folder; if loading fails, use llama.cpp or merge them withllama-gguf-split --merge.