This post assumes the block quantisation, K-quants, and imatrix background from part 1.
Dynamic quantisation (v1) — model-specific, but MoE-focused
Unsloth's first "Dynamic" quants targeted mixture-of-experts models like DeepSeek-R1. MoE architectures have uneven sensitivity across their many experts — some experts are activated constantly and need precision, others are rarely activated and can be compressed hard with little damage. A uniform K-quant rule spends bits evenly across experts regardless of that difference. Dynamic v1 measured per-expert importance and allocated bits accordingly, staying mostly within MoE models.
Dynamic 2.0 — generalised, per-layer, dense models included
Dynamic 2.0 (the source of the UD-Q4_K_XL files) extended this to every layer of every model, including dense architectures — Llama 4, Gemma 3, Qwen3.5, Phi-4.
- Per-layer analysis, not a per-tensor-type rule. Instead of "always bump
ffn_downtoQ6_K," Dynamic 2.0 analyses each layer's sensitivity via imatrix-style calibration and sets its precision individually. Most weights still land atQ4_K, but specific attention layers, embeddings, and output projections get pushed toQ5_K/Q6_Kwhere the calibration signal supports it — and sometimes a layer that the staticQ4_K_Mrule would bump does not get bumped, if the calibration data says that layer in that model isn't sensitive. - Model-specific schemes. Because the analysis runs per model, the resulting scheme differs — the layers upcast in a Gemma 3 build are not the same set as in a Llama 4 build.
- New calibration data. Unsloth used roughly 1.5M+ tokens of hand-cleaned conversational data for the imatrix step, on the basis that calibration data composition changes which activations get flagged as important.
- New low-level formats for edge hardware:
Q4_NL,Q5.1,Q5.0,Q4.1,Q4.0, tuned for Apple Silicon/ARM.
Naming: UD-Q4_K_XL — "UD" marks it as Unsloth Dynamic rather than stock llama.cpp K-quants, and "XL" replaces the S/M/L sizing to indicate the per-layer allocation at roughly the Q4 tier.
How the accuracy claim is measured. Unsloth argues perplexity, the standard quant-quality metric, is a poor measure: aggregate perplexity can stay flat even while individual token predictions flip from correct to incorrect and back, because the errors cancel out across the dataset. They report KL-divergence against the unquantized model's output distribution instead, which is more sensitive to those flips. They've also published a case where a quant scoring worse on both perplexity and KLD (IQ2_XXS) beat a competing quant with better numbers (IQ3_S) on downstream evals like LiveCodeBench and MMLU Pro — evidence that KLD, while better than perplexity, is also not a complete measure.
Dynamic 3.0 — further layer selection, and a new metric
Dynamic 3.0, released within the last few weeks:
- New calibration dataset, composed for agentic coding, chat, and multilingual performance rather than generic text.
- A new metric, "Divergence-300@32." 300 holdout prompts (from Terminal-Bench 2.1, DeepSWE, MathArena, and similar — excluded from calibration data) are run through both the BF16 model and the quantised model with greedy decoding, and divergence is measured across a 32-token generated sequence rather than a single token. This extends the "does the quantised model still agree with the original" check from one token to a short trajectory, closer to actual task behaviour than single-token perplexity or KLD.
- Post-training quantisation only — no QAT (quantisation-aware training) or QA-distillation; everything derives from the trained BF16 weights plus the calibration pass.
- Sub-2-bit variants, e.g.
UD-IQ1_S, reported to retain most top-1 accuracy at a fraction of the size. Very small quants also drop the MTP (multi-token-prediction) module to save disk space, with MTP available as a separate add-on. - Reported result: >10% better top-1 accuracy at matched file size versus other providers' quants, on their Qwen3.8-27B release.
Decoder ring: reading a quant filename
| Segment | Meaning |
|---|---|
| Q4 / Q5 / Q8 | Nominal bits per weight for most of the model |
| _0 / _1 | Old-style block quant: _0 = scale only, _1 = scale + zero-point |
| _K | K-quant superblock scheme (256-weight superblocks, 8×32 sub-blocks) |
| _S / _M / _L | Static, fixed rule for which tensor types get bumped up |
| UD-... | Unsloth Dynamic: per-layer, model-specific allocation via calibration |
| _XL / _XXL | Dynamic-scheme sizing tier, produced by per-layer analysis rather than a static rule |
| IQ2 / IQ1 etc. | Importance-matrix quant — extreme low-bit formats that rely on calibration data to decide where to spend precision, since there is not enough bit budget for a naive scheme to work |
Practical selection guide
- If a Dynamic (UD) build exists for your model, prefer it over the equivalent stock K-quant at the same nominal bit-width — the accuracy gap is largest at Q4 and below.
- The gap narrows going up the precision ladder. Reports put Dynamic's advantage as largest at Q2–Q4 and small by Q5–Q6, where both approaches are close to lossless. If
Q5_K_Mfits your VRAM, the standard format is a reasonable choice. - Small dense models (2B–4B) show a smaller absolute gain than the 27B–70B+ models most benchmarks are run on. Worth checking against your own task rather than assuming reported percentage gains transfer down linearly.
- File size does not reliably indicate which is better. A
UD-Q4_K_XLand stockQ4_K_Mcan come out close in size, or either can be smaller, because the per-layer decisions push different amounts of the model up or down.