README.md

---
license: other
license_name: swift-open-license-1.0
license_link: LICENSE
library_name: vllm
base_model:
  - ukisai/Swift-Qwen3.8-27b
  - TheUnderscore/Swift-Qwen3.8-27b-W4A16-AWQ
tags:
  - qwen
  - qwen3.8
  - swift
  - w4a16
  - gptq
  - compressed-tensors
  - speculative-decoding
  - mtp
  - rtx-3090
  - syv
pipeline_tag: text-generation
---

# Swift-Qwen3.8-27B-W4A16-syv-fast

The [syv-ai/qwen38-27b-rtx3090](https://github.com/syv-ai/qwen38-27b-rtx3090)
**fast-variant serving shape** of
[ukisai/Swift-Qwen3.8-27b](https://huggingface.co/ukisai/Swift-Qwen3.8-27b)
(the "reduced reasoning" finetune of Qwen3.8-27B), built entirely from
Swift's own weights and outputs:

- **Body**: TheUnderscore's W4A16 asymmetric-AWQ g128 compressed-tensors
  quant of Swift, carried through unmodified.
- **lm_head + MTP**: int4 GPTQ (g128, symmetric) calibrated on **Swift's own
  hidden states** — 300k rows captured through the syv `drafter/` pipeline.
  lm_head KL to the bf16 head: **0.00234** (RTN: 0.00707; the official syv
  Qwen fast variant shipped at 0.0029). MTP relative errors 0.146–0.176,
  inside the range of the shipped syv fast variant.
- **Embeddings**: int8 g128, requantized from Swift's own weights.
- **Draft vocab**: counted over 4.23M tokens of **Swift's own outputs** on a
  coding-agent-weighted prompt corpus (35% Rust, 16% TypeScript, 11%
  debugging, 10% code-edit, 8% agent/tool-use, 10% architecture, 7%
  technical reasoning, 3% general; 61% thinking-on). Swift emits far fewer
  distinct tokens than base Qwen (~25.9k vs ~54k), so the draft head is
  25,879 rows, not the base model's 40,960.

Held-out coverage (10% of sequences, never counted against):

| draft vocab | all tokens | code sources |
|---|---|---|
| **Swift-derived (this model)** | **99.81%** | **99.86%** |
| base-Qwen 40k list | 96.69% | 96.67% |

## Serving

Designed for the syv single-user stack (vLLM 0.28.0 + the repo's patch
series — the MTP draft-vocab patch is required for the 25,879-row head;
stock vLLM will not use it):

```bash
MODEL=/path/to/Swift-Qwen3.8-27B-W4A16-syv-fast \
  SPEC=mtp PREFIX_CACHE=1 CTX=long MAX_LEN=114688 \
  bash single-user/start_qwen.sh    # from the syv checkout
```

The syv `verify.sh` will report one false FAIL on this dir (it asserts an
int8 lm_head; this is int4 by construction, like the official fast variant).

## Measured (RTX 3090, syv stack, MTP + prefix caching, 114,688 context)

| | decode tok/s | MTP acceptance | tok/step |
|---|---|---|---|
| **this model** | 98.4 | **0.660** | 2.98 |
| Swift + int8 heads, base draft vocab | 94.0 | 0.630 | 2.89 |
| official syv Qwen fast variant | 98.2 | 0.634 | 2.90 |

Quality battery (identical prompts): 8/9, matching the int8 build — tool
calling, strict JSON, streaming and the qwen3 reasoning parser all clean.
Swift's reasoning-termination behaviour is preserved (GPTQ heads verified
not to shift it, including under greedy decoding).

## Provenance & licence

Attribution chain: **Alibaba Cloud** Qwen3.8-27B (Apache-2.0, included as
`LICENSE-APACHE-2.0`) → **UkisAI** Swift finetune (Swift Open License v1.0,
included as `LICENSE`) → **TheUnderscore** W4A16-AWQ body (same licence) →
this repository's int4-GPTQ heads, int8 embeddings and Swift-derived draft
vocabulary (quantisation and calibration by **liamwh**, using the
[syv-ai/qwen38-27b-rtx3090](https://github.com/syv-ai/qwen38-27b-rtx3090)
`drafter/` pipeline with a coding-agent-weighted Swift corpus). Distributed
under the Swift Open License v1.0; commercial use above its revenue
threshold requires the Swift Enterprise License.

Rebuild from scratch with the syv checkout's `drafter/` pipeline
(`gen_data.py` → `capture.py` → GPTQ heads → `build_draft_vocab.py`) over a
Swift-weighted prompt corpus; `swift_draft_vocab_ids.json` here is the
exact id list this model serves.