Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast

Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM prepared for HyperQwen — Qwen3.8-27B served fast on one 24 GB card — the way the TurboQwen image expects it. This is the uncensored (abliterated) finetune; its behaviour is inherited from orcarouter/Qwen3.8-27B-Uncensored. The body is untouched: int4 asymmetric AWQ, group 128, zero points, quantized with llm-compressor from bf16 weights; the vision tower, the SSM gate projections and the MTP head's norms stay bf16. What changed is what HyperQwen's prepare/ pipeline changes, so that a 24 GB card has room for a KV cache and speculative decoding has something small to score:

tensor	in the source export	here
lm_head	bf16, 2.5 GB	int4 GPTQ (group 128, symmetric), calibrated on 400k of the model's own final hidden states (drafter/gptq_lm_head.py; RTN int4 KL 0.0070 -> GPTQ 0.0024 against the bf16 head)
embed_tokens	bf16, 2.5 GB	int8 (group 128, symmetric), round-trip error 0.65%
MTP module (mtp.*, 8 linears incl. mtp.fc)	bf16, 850 MB	int8 (group 128, symmetric)
mtp.draft_lm_head + mtp_draft_vocab_ids.pt	—	40,960-row draft head (int4, sliced from the int4 lm_head), HyperQwen's shipped id list
body (64 layers), vision tower	int4 asym g128 / bf16	unchanged

Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen keeps lm_head in int8 instead: +0.6% perplexity less, a few tok/s less.

Serving

The container does everything (download, verify, serve), with the vision tower on:

git clone https://github.com/Ar4ikov/TurboQwen && cd TurboQwen
cp .env.example .env            # CHECKPOINT=uncensored   and MODEL=/app/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast
docker compose --profile single up -d

Bare metal, on the fork branch this was measured with (Ar4ikov/HyperQwen@awq-asym, vLLM 0.29.0 + the series + marlin-int8-asym-zp):

hf download Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast --local-dir models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast
VISION=1 MODEL=$PWD/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast SPEC=mtp CTX=fast bash single-user/start_qwen.sh

Plain vLLM 0.29 also loads it (vllm serve Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast --max-model-len 65536), without the speculative decoding, the draft head or the int8 Marlin path that need the patch series.

Measured

RTX 3090 (350 W), vLLM 0.29.0, HyperQwen bench/run_benchmarks.sh single, second run after boot, VISION=1 (the tower streamed from host RAM per image). C1 = one stream of real prompts with 1,024-token answers; tok/step = tokens accepted per forward pass.

profile	C1 T=default	C1 T=0	tok/step	C8 T=default	KV pool
SPEC=mtp CTX=fast (64k, bf16 KV)	112.7 tok/s	127.6	2.74 / 2.95	434 tok/s	80,185
SPEC=dflash2 CTX=fast KV_MEM=4600000000 DFLASH_MAX_LEN=49152	135.9	139.3	3.29 / 3.29	401	53,233
production line: DFlash2 k=15 + INT8_ACT=int8 PREFILL_ATTN=int8 (40k; TTFT 96 ms)	116.7	135.5	2.97 / 3.44	n/a (4 slots)	42,113

Images are described correctly in every profile (a drawn red square, blue circle and a line of text: boost/image_smoke.py). The whole table, the int8 (W4A8) profiles and the kernel measurements: github.com/Ar4ikov/TurboQwen.

Why a separate repo

HyperQwen's pipeline rewrites the checkpoint in place (int8 heads, the draft head) and its launchers, verify script and Docker entrypoint expect that layout. Doing it once and publishing the result turns a ~10-minute CPU step per machine into a download, and keeps the original export as it is for transformers and plain vLLM users.

Quantization recipe of the body, calibration set and the AWQ mappings for the hybrid Gated-DeltaNet / attention layers: see the source export's card and its recipe.yaml (kept here). Root model: Qwen/Qwen3.8-27B, Apache-2.0.