vmarcelo/Swift-Qwen3.8-27B-MIX_GGUF

⚠️ Superseded — use Swift 1.5 instead

This quant has been superseded by ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF, the official GGUF release of Swift 1.5. I recommend using that release instead — this page remains available as an archive.

---

Swift-Qwen3.8-27B — MIX GGUF (IQ4 + IQ3 + IQ2)

Three custom mixed-tensor quantizations of ukisai/Swift-Qwen3.8-27b, a fine-tune of Qwen/Qwen3.8-27B — the dense 27B multimodal model with hybrid GatedDeltaNet + Gated Attention and a Multi-Token Prediction (MTP) head.

Built with llama.cpp 58367713a, using a purpose-built importance matrix and per-tensor quantization overrides. All three are designed around the 16 GB VRAM budget of consumer GPUs such as the AMD Radeon RX 9070 XT, and all keep the MTP head embedded so speculative decoding works out of the box — no second file to download or load.

Files

File	Size	BPW	Category	Best for
Swift-Qwen3.8-27B-IQ4-MIX.gguf	14.16 GB	4.15	Q4	Best quality that still fits 16 GB at moderate context
Swift-Qwen3.8-27B-IQ3-MIX.gguf	12.87 GB	3.77	Q3	Long context (64K–128K), agentic workloads, VRAM headroom
Swift-Qwen3.8-27B-IQ2-MIX.gguf	10.12 GB	2.96	Q2	Maximum context (131K–262K) on 16 GB; experimental quality tier
mmproj-F16.gguf	889 MB	—	—	Vision encoder — load with --mmproj
swift_iq4_types.txt	3.4 KB	—	—	IQ4-MIX per-tensor recipe
swift_iq3_types.txt	3.4 KB	—	—	IQ3-MIX per-tensor recipe
swift_iq2_types.txt	3.4 KB	—	—	IQ2-MIX per-tensor recipe
swift-imatrix.gguf	14 MB	—	—	Importance matrix used for all builds (provenance)
config.json	3.8 KB	—	—	Arch metadata for LM Studio / HF Hub

Recipes

Both quants follow the same philosophy: protect what the model cannot afford to lose, and pay for it out of the FFN, which is two-thirds of the file.

IQ4-MIX — 4.15 BPW

Tensor family	Type	Size	Share
FFN down	IQ4_XS, ends Q5_K, blk.63 Q6_K	3.27 GB	23.1%
Linear attention (qkv / gate / out)	IQ4_XS, gate Q5_K	3.18 GB	22.5%
FFN up	IQ3_XXS mid, IQ4_XS ends, blk.63 Q6_K	2.42 GB	17.1%
FFN gate	IQ3_XXS everywhere, blk.63 Q4_K	2.20 GB	15.5%
Full attention (q/k/v/o)	IQ4_XS q, Q8_0 k+v, Q5_K o	1.06 GB	7.5%
output.weight	Q5_K	0.87 GB	6.2%
token_embd	IQ4_XS	0.68 GB	4.8%
MTP (blk.64)	Q8_0	0.45 GB	3.2%
ssm_alpha / ssm_beta	Q8_0	0.03 GB	0.2%
norms, ssm_a, ssm_conv1d	F32	<0.01 GB	<0.1%

IQ3-MIX — 3.77 BPW

Tensor family	Type	Size	Share
Linear attention (qkv / gate / out)	IQ3_XXS mid, IQ4_XS ends, blk.0 Q5_K	2.70 GB	21.0%
FFN down	IQ4_XS ends, IQ3_XXS mid, blk.63 Q5_K	2.52 GB	19.5%
FFN up	IQ4_XS ends, IQ3_XXS mid, blk.63 Q5_K	2.52 GB	19.5%
FFN gate	IQ3_XXS everywhere, blk.63 Q4_K	2.20 GB	17.1%
Full attention (q/k/v/o)	IQ3_XXS q, Q5_K k+o, Q6_K v	0.87 GB	6.8%
output.weight	Q5_K	0.87 GB	6.8%
token_embd	Q4_K	0.72 GB	5.6%
MTP (blk.64)	Q8_0	0.45 GB	3.5%
ssm_alpha / ssm_beta	Q8_0	0.03 GB	0.2%
norms, ssm_a, ssm_conv1d	F32	<0.01 GB	<0.1%

IQ2-MIX — 2.96 BPW

Tensor family	Type	Size	Share
Linear attention (qkv / gate / out)	IQ4_XS gate+out, IQ2_S qkv mid / IQ3_XXS ends	2.47 GB	24.4%
FFN down	IQ2_XS mid, IQ3_XXS ends, blk.63 Q5_K	1.88 GB	18.6%
FFN up	IQ2_XXS mid, IQ3_XXS ends, blk.63 Q4_K	1.75 GB	17.4%
FFN gate	IQ2_XXS everywhere, blk.63 Q4_K	1.50 GB	14.8%
Full attention (q/k/v/o)	IQ3_XXS q, Q6_K k+v, IQ4_XS/IQ3_XXS o	0.87 GB	8.6%
output.weight	Q5_K	0.87 GB	8.6%
token_embd	IQ3_XXS	0.49 GB	4.8%
MTP (blk.64)	Q4_0	0.24 GB	2.4%
ssm_alpha / ssm_beta	Q8_0	0.03 GB	0.2%
norms, ssm_a, ssm_conv1d	F32	<0.01 GB	<0.1%

Built on a deliberate inversion. The FFN is 62.6% of all parameters and is the only place with enough mass to fund a 10 GB file, so it absorbs the cuts (IQ2_XXS mid-block through IQ2_XS), while everything the sensitivity research flags as fragile stays high — including attn_k/attn_v at Q6_K and output at Q5_K, which together are under 1 GB. The consequence is that linear attention becomes the largest family (24.4%) rather than the FFN. The FFN's 62.6% share is why a 2-bit tier is even possible: cutting the FFN by 1.5 bits buys about 3.2 GB, more than enough to keep every small sensitive tensor near-lossless.

Expect a real quality cost versus IQ3. Published 2-bit vs 3-bit comparisons for this family show roughly 5 points of top-1 and ~3× the KLD, so treat this as the maximum-context tier rather than a general-purpose one. It does still reason coherently — on a smoke test with a long mixed-topic prompt it correctly identified the actual question buried at the end.

Why these choices

• ssm_alpha / ssm_beta at Q8_0 (~96 tiny tensors) — they gate the DeltaNet path. These are held at Q8_0 in every published tier of this family from 1-bit to 8-bit, because crushing them (uniform Q3_K was measured as the worst configuration tried) inflates KLD far beyond the disk it saves. They are 0.2% of the file — never worth touching.
• ssm_out protected (IQ4_XS/Q4_K+) — reported as the single most sensitive tensor in hybrid models (~6.0 KLD at 2-bit). It is cheap to protect because it is small.
• attn_gate and attn_qkv kept high (Q5_K/IQ4_XS) — quantizing any attention tensor is more damaging in hybrid architectures than in pure-attention models. Earlier recipes that pushed linear-attention projections to IQ2_S were the main quality loss in that generation.
• Full attention split, not uniform — attn_k/attn_v get Q8_0 (IQ4-MIX) or Q6_K (IQ3-MIX) because they are tiny (7% of the file combined) and highly sensitive, while attn_q drops to IQ4_XS/IQ3_XXS where it is cheap.
• FFN gate is the funding source — it never earns extra bits in any sensitivity sweep, so it sits at IQ3_XXS in both files, including the Q4. ffn_up takes a partial cut mid-block only, and ffn_down is the most protected FFN tensor (it is the most sensitive of the three).
• Ends lifted — first 4 (IQ4) / 12 (IQ3) blocks and the last 12 blocks are raised one notch, with blk.63 raised further. Per-layer sensitivity is U-shaped.
• MTP at Q8_0, embedded — the importance matrix never exercises the MTP block, so it receives no importance weighting; Q8_0 is the safe near-lossless bound for 0.45 GB. Embedding it costs 0.45 GB versus a ~1.9 GB standalone draft file, because a standalone draft must duplicate the 248K-vocab embedding and output head (2.54B params) to function.

How these were built

Source. The Swift BF16 safetensors were quantized directly — not the FP8 release, which would double-quantize.

1. Convert to GGUF F16 (convert_hf_to_gguf.py --outtype f16), MTP retained: 866 tensors, 54.6 GB. Removing MTP is not done with llama-quantize --prune-layers — on this architecture that drops the tensors but leaves qwen35.attention.recurrent_layers at 65 entries and the file will not load at all. MTP removal must happen at conversion time (--no-mtp).

2. Build the importance matrix. The calibration corpus is a chat-template-rendered mixture of prose and tool-calling turns (214 prose + 369 tool conversations, 583 chunks at ctx 512), tokenized with --parse-special so the template's special tokens are handled correctly.

Because a 27B model must be fully resident to calibrate in reasonable time, the imatrix was computed on a 13 GB IQ4 copy of the model rather than the 51 GB F16: the F16 spills out of page cache on a 32 GB machine and re-reads 31 GB from disk per chunk (18 TB total), and a Q6_K copy offloaded only 36/65 layers and ran at 36.8 s/pass (ETA ≈ 6 h). With the IQ4 copy fully in VRAM (-ngl 99) the run took 23.6 minutes at 2.44 s/pass, 0 CPU layers.

3. Quantize with llama-quantize --imatrix swift-imatrix.gguf --tensor-type-file ….

4. Validate. The generated importance matrix was diffed against an independently published imatrix for the same model and corpus: 496/496 tensors shared, cosine similarity median 0.999997, mean 0.999746, minimum 0.995322, zero tensors below 0.99 — confirming that calibrating on a 4-bit copy produces statistics equivalent to a 16-bit one.

Both GGUFs were then load- and generation-tested on an RX 9070 XT (Vulkan, -ngl 99, 512 ctx, temperature 0).

Usage

# llama.cpp — desktop / server
llama-server -m Swift-Qwen3.8-27B-IQ4-MIX.gguf \
  --mmproj mmproj-F16.gguf \
  -ngl 999 -c 16384 -b 2048 -ub 2048 -t 12 -np 1 \
  --cont-batching --jinja --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 --kv-unified \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 \
  --host 0.0.0.0 --port 8080

The MTP head is embedded, so speculative decoding needs only --spec-type draft-mtp — there is no --model-draft file to pass. Drop that flag (and --spec-draft-n-max) for a correctness-first baseline run.

RADV_PERFTEST=nogttspill is recommended on RDNA4 with RADV to avoid GTT spill.

Measured performance (RX 9070 XT, Vulkan, RADV)

32K context, MTP on, -b 2048 -ub 512, temperature 0, 100 KB prompt (~22.5K tokens) so the KV cache is genuinely deep. IQ3-MIX is 2 runs; IQ2-MIX and IQ4-MIX are 4 runs (mean ± stdev):

Model	KV type	Prompt	Generation	Peak VRAM
IQ2-MIX	Q8_0/Q8_0	989.5 ±17.6 t/s	58.1 ±1.9 t/s	11,766 MiB
IQ2-MIX	Q5_0/Q4_1	878.8 ±6.6 t/s	58.8 ±1.1 t/s	11,258 MiB
IQ3-MIX	Q8_0/Q8_0	924.7 t/s	55.7 t/s	14,118 MiB
IQ3-MIX	Q5_0/Q4_1	828.2 t/s	49.0 t/s	13,610 MiB
IQ4-MIX	Q8_0/Q8_0	913.2 ±46.4 t/s	48.5 ±1.6 t/s	15,432 MiB
IQ4-MIX	Q5_0/Q4_1	827.9 ±11.5 t/s	52.5 ±2.4 t/s	14,928 MiB

Keep Q8_0 for the KV cache.

• Prompt processing is consistently faster with Q8_0 in every tier, by 9–13%. This is the one reliable effect and it points the same way each time.
• Generation is inconsistent between tiers, with no clean trend: Q5_0/Q4_1 costs IQ3-MIX 12%, is a wash on IQ2-MIX (+1%), and actually helps IQ4-MIX by ~7%. It never wins on prefill, so it is not a performance option — you cannot predict which way it will land.
• The saving is 508 MiB at 32K in every case, and scales with context.

A control run with RADV_PERFTEST=nogttspill — testing whether IQ4-MIX's 94.6%-of-card VRAM peak was spilling into GTT — changed nothing beyond noise, so IQ4-MIX's different behaviour is real rather than a spill artifact.

Treat Q5_0/Q4_1 as a capacity lever only, not a performance one. At 131K the same swap frees ~1.6 GiB, which is where it earns its keep — and you pay ~10% of prompt speed for it.

One practical note: IQ4-MIX at 32K with Q8_0 KV sits at 94.6% of the card, so it wants an otherwise-idle GPU. If your desktop is using VRAM, drop to Q5_0/Q4_1 for the headroom or use IQ3-MIX.

Speculative decoding is a large win and costs nothing to enable: the MTP head is embedded, so --spec-type draft-mtp alone raises generation from ~33 t/s to ~56 t/s on the same recipe.

Context guidance (16 GB VRAM)

Only 16 of the 65 blocks are full attention, so the KV cache is small: 64 KiB per token at F16, 34 KiB at Q8_0, 21 KiB at Q5_0/Q4_1, 18 KiB at Q4_0. Measured resident size: 13.8 GiB for IQ3-MIX and 11.5 GiB for IQ2-MIX, both at 32K context with Q8_0/Q8_0, against 15.9 GiB of usable VRAM.

Context	KV type	IQ4-MIX (14.2 GB)	IQ3-MIX (12.9 GB)	IQ2-MIX (10.1 GB)
32K	Q8_0	workable	comfortable (13.8 GiB)	easy (11.5 GiB)
64K	Q8_0	tight	workable	comfortable
131K	Q8_0	not viable	not viable	works
131K	Q5_0/Q4_1	not viable	borderline — needs a free GPU	comfortable
262K	Q5_0/Q4_1	not viable	not viable	works (native max)

IQ4-MIX for best quality at moderate context · IQ3-MIX for long context · IQ2-MIX when you need 131K+ on a 16 GB card and accept the quality cost.

Rule of thumb from the measurements: budget ~1.2 GiB of fixed overhead (compositor, compute buffers, the 0.15 GiB recurrent state) plus KV, and keep the total under 15.9 GiB.

Model notes

• Dense 27B, 64 text layers + 1 MTP layer, 48 GatedDeltaNet linear-attention layers and 16 full-attention layers (every 4th), hidden 5120, vocab 248,320, native context 262,144, and a vision encoder.
• Swift is a fine-tune that suppresses overthinking: 58.3% fewer thinking tokens, <1% average accuracy loss, ~1.95× faster. Deltas vs base: GPQA-D 88.38→88.28, MMLU-Pro 85.47→84.95, IFBench 73.53→71.80, AIME 98.67→94.00, HMMT 99.33→96.00, LCBv6 76.76→81.55, TB2.1 66.74→65.84.
• Thinking is on by default; reasoning_effort accepts xhigh (default), medium, low.

Credits

• Model: ukisai/Swift-Qwen3.8-27b by UkisAI — the fine-tuned weights these GGUFs were built from.
• Base model: Qwen/Qwen3.8-27B by the Qwen team.
• Vision projector: derived from the same checkpoint's vision encoder.
• Calibration corpus: bartowski/ukisai_Swift-Qwen3.8-27b-GGUF (calibration-v6 chat-template mixture), whose published imatrix was also used as the independent QC reference.
• Quantization toolchain: llama.cpp at commit 58367713a.
• Recipe research: tensor-sensitivity guidance from Unsloth's Qwen3.5 hybrid benchmarks, bartowski's per-tensor layout work, and published layouts from AtomicChat and ukisai.

License

The base model is Apache-2.0. The Swift contribution is released under the Swift Open License v1.0 — free for personal, research, educational and evaluation use, and free for commercial use while gross annual revenue (including affiliates) is at or below US$1M; above that an enterprise licence from UkisAI is required. These GGUF conversions inherit those terms. Quantized by vmarcelo.