signalnine/Qwen3.6-27B-MTP-q27

Qwen3.6-27B-MTP -- q27 format (seven tiers, 4.55-8.1 bpw)

Repack of vanilla Qwen3.6-27B-MTP (hybrid GDN+attention with trained-in MTP heads) for the q27 inference engine -- a from-scratch CUDA C++ engine targeting single consumer GPUs (RTX 5090, with sm_86 fallback), built around MTP self-speculation with an adaptive draft-depth ladder.

Files

file	md5	size
qwen36-27b-mtp.q27	39fd724426088abaef6e0bad66d6dac6	17.73 GB
qwen36-27b-mtp.tok	bb95b3ca7647ce1cc061c141789e7102	7.2 MB

Quant recipe: v1.4 repack, Q4 blocks with q8 on (ssm_out|attn_output) tensors (--q8 '(ssm_out|attn_output)\.'), 5.25 bits/weight overall. Converted from the merged BF16 of unsloth/Qwen3.6-27B-MTP-GGUF.

Usage

git clone https://github.com/signalnine/q27 && cd q27 && make
# CLI
./build/q27 qwen36-27b-mtp.q27 --tokens "760,6511,314,9338,369" --ctx 2048 --spec -n 128
# Server (Anthropic /v1/messages + OpenAI-compatible endpoints)
Q27_KV=fp8 Q27_PMIN=0.5 Q27_MAXD=auto Q27_DEXIT=1 \
  ./build/q27-server qwen36-27b-mtp.q27 qwen36-27b-mtp.tok \
  --port 8081 --ctx 131072 --no-think --fast-head

Production recommendation: Q27_KV=fp8 Q27_PMIN=0.5 Q27_MAXD=auto Q27_DEXIT=1 --fast-head.

Determinism

Greedy decode is bitwise-deterministic per GPU architecture and width-invariant across the spec-depth ladder. Canonical gate (the command above, md5 of the generated: line):

• RTX 5090 (sm_120): a2982c5197c627551b27d76a0a94b220
• RTX 3090 (sm_86, fp16 KV): 6894254e3b1a184ee3802771ddd59c2b

Cross-architecture identity is NOT guaranteed (near-tie logits fork on codegen/ULP differences); identity holds within an architecture.

Performance (RTX 5090, 2026-07-09, engine master 197d6b6)

• Decode 160-190 t/s at ~26K context by flavor; live Claude Code traffic aggregates ~168 t/s (5.3 tok/round via the adaptive depth ladder).
• Prefill (fp8 batched): 8K in 2.35s, 32K in 10.4s, 128K in 59.4s (2206 t/s).
• 95% prompt-token prefix-cache hit rate under agentic serving.
• RTX 3090: ~60 t/s decode at 26K, 32K context ceiling on 24 GB.

Provenance

Qwen3.6-27B-MTP by Alibaba/Qwen (Apache-2.0), GGUF export by unsloth, q27 repack by signalnine. The fine-tune variant Qwopus3.6-27B-v2-MTP-q27 runs ~6% faster on real agentic traffic (higher MTP acceptance; the old ~35% figure was an echo-heavy replay best-case, revised 2026-07-11).

q6 tier (quality-first, 5090-class)

qwen36-27b-mtp-q6.q27 -- 20.49 GB, 6.0 bits/param. Same v1.4 policy plus every ffn_down tensor promoted to Q8_G128 (the one promotion the sensitivity study found worth taking whole; GDN in-projections stay Q4 on purpose -- promoting them measures worse).

Measured (matched protocol, RTX 5090, 2026-07-12):

	5.25 bpw default	q6	Q5_K_M bar (llama.cpp)
wikitext-2 PPL	8.0409 (+1.55%)	7.9460 (+0.35%)	7.9179
short-ctx decode	171.5 t/s	152.1 (-11%)
26K-deep decode	176.6 t/s	169.2 (-4%)
fp8 auto-ctx (32GB)	262144	196608

Not for 24GB cards: the fixed cost alone (weights + role sets + graph zoo) is ~24.2 GB. On a 3090, use the default artifact.

md5: 7a0d21b4e44254e1820ca4fd75ea8fee (also in CHECKSUMS.md5). Requires q27 server built after 2026-07-12 (auto-ctx sizes from the model file; older builds OOM at boot on this tier with fp8 KV).

q6k tier (GGUF-flagship quality, 5090-class)

qwen36-27b-mtp-q6k.q27 -- 23.25 GB, 6.8 bits/param. q6's policy plus every ffn_gate tensor promoted to Q8 (ffn_up is deliberately NOT promoted: it measures WORSE -- its Q4 noise cancels inside the SwiGLU product).

Matched-protocol wikitext-2 PPL (RTX 5090, 2026-07-12): 7.9127 -- below unsloth Q5_K_M (7.9179), Q6_K (7.9811), and statistically indistinguishable from the 26 GB UD-Q6_K_XL flagship (7.9584 +/- 0.061) at 2.75 GB smaller. Decode: 143 t/s short-ctx / 151 t/s at 26K depth (vs 172/177 for the 5.25 bpw default). Auto-ctx on 32 GB: fp8 114688, turbo3 262144. Not for 24 GB cards.

q4s tier (max context, small cards)

qwen36-27b-mtp-q4s.q27 -- 15.46 GB, 4.55 bits/param. The v1.4 policy minus the promotions that don't pay: one Q4 lm_head serves draft, verify, and plain decode (the Q8 head and its Q4 duplicate are gone), and ssm_out/attn_output revert to Q4. token_embd, the MTP layer, and attn k/v stay Q8. Needs a q27 build from 2026-07-16 or later (repack.py --q4-head; older engines work too -- the format is unchanged, only the policy).

Matched-protocol wikitext-2 PPL (RTX 5090, 2026-07-16, paired on the same 148,335 predictions): 8.0197 vs the default tier's 8.0409 -- the smaller tier measures 0.26% BETTER (error cancellation: the same structure that makes ffn_up promotion hurt). Decode: 186 t/s short-ctx suite vs 177 for the default, same day. The 2.27 GB it returns is ~167K tokens of turbo3 KV budget: sub-24GB cards (A10 class, ~22.6 GiB usable) go from a ~28K ctx ceiling to ~200K+, and a dedicated 24 GiB 3090 reaches the 262144 auto-ctx cap. Canonical: f64e7c02252ca4c40cea62db662205e0.

md5: see CHECKSUMS.md5.

q5f tier (best quality on 24 GB)

qwen36-27b-mtp-q5f.q27 -- 18.22 GB, 5.30 bits/param. q4s's single-Q4 lm_head plus every ffn_down tensor promoted to Q8. The promotions STACK: the Q4 head's error cancellation survives FFN promotions but not attn/ssm ones (promoting ssm_out/attn_output on the Q4 head measures WORSE than plain q4s). Matched-protocol wikitext-2 PPL (2026-07-22): 7.9491 -- beats q4s (8.0197), the default (8.0409), and q8 (7.9942); matches q6 (7.9460) at 2.3 GB less. HumanEval+ 30/30, LiveCodeBench 23/30 (>= q4s on both). On a 24 GB 3090 auto-ctx lands ~69632 vs q4s's 262144 cap -- the quality-over-context pick. Canonical: 683f7f4450ca4c60837abdb603ee3237.

q6f tier (the 32 GB pick)

qwen36-27b-mtp-q6f.q27 -- 20.99 GB, 6.11 bits/param. q5f plus every ffn_gate tensor promoted to Q8. Matched-protocol wikitext-2 PPL (2026-07-22): 7.9189 -- beats q6 (7.9460) at the same size and sits near q6k (7.9127) at 2.25 GB less. HumanEval+ 30/30 (q6: 29/30), LCB 22/30 (tied). Auto-ctx 184320 (fp8) on a 32 GB 5090. Supersedes q6 as the 32 GB recommendation. Canonical: 2a4d22eafcde63e962bf2408605fe502.

Tier map: q4s 4.55 bpw (max context, fastest) | default 5.25 bpw (reference, the most-measured configuration) | q5f 5.30 bpw (best quality that fits 24 GB) | q6 6.0 bpw (superseded by q6f) | q6f 6.11 bpw (the 32 GB pick) | q6k 6.8 bpw (flagship quality).