---
license: apache-2.0
base_model: z-lab/Qwen3.5-9B-DFlash
tags:
- gguf
- llama.cpp
- speculative-decoding
- dflash
- draft-model
---

# Qwopus3.5-9B-Coder-DFlash-GGUF

DFlash draft model for [Jackrong/Qwopus3.5-9B-Coder](https://huggingface.co/Jackrong/Qwopus3.5-9B-Coder), for use with upstream llama.cpp speculative decoding (`--spec-type draft-dflash`, merged in [#22105](https://github.com/ggml-org/llama.cpp/pull/22105)).

This is [z-lab/Qwen3.5-9B-DFlash](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash) converted with `--target-model-dir` pointing at the Qwopus tokenizer. Qwopus extends the Qwen3.5 tokenizer with 7 added tokens (ids 248070 to 248076), so drafts converted against the base Qwen3.5-9B tokenizer do not pass the vocab compatibility check for this target. The draft GGUF carries no token embeddings or lm_head; llama.cpp shares the target model's at runtime, so the draft matches whatever Qwopus quant you serve.

Conversion and launch scripts: https://github.com/Gaurav-Gosain/qwopus-dflash

<video controls src="https://huggingface.co/GauravGosain/Qwopus3.5-9B-Coder-DFlash-GGUF/resolve/main/demo.mp4"></video>

The demo replays two real captured token streams applying the same edit to a Go file (same prompt, temperature 0): baseline left, DFlash right.

## Usage

```sh
llama-server \
  -hf Jackrong/Qwopus3.5-9B-Coder-GGUF:Q3_K_M \
  -hfd GauravGosain/Qwopus3.5-9B-Coder-DFlash-GGUF:Q4_K_M \
  --no-mmproj \
  --spec-type draft-dflash --spec-draft-n-max 15 \
  -fa on --jinja -c 4096 -ctk q8_0 -ctv q8_0 -ctxcp 2 -fitt 256
```

Pass `-hf` and `-hfd` together (local `-m` plus `-hfd` currently fails to resolve the draft) and keep `--no-mmproj` (the target repo ships a 921 MB vision projector).

Qwen3.5 is a hybrid linear-attention architecture; keep `-ctxcp` low because each context checkpoint stores the full recurrent state (about 100 MB).

## Files

| file | size | note |
| --- | --- | --- |
| Qwopus3.5-9B-Coder-DFlash-Q4_K_M.gguf | 766 MB | recommended |
| Qwopus3.5-9B-Coder-DFlash-Q8_0.gguf | 1.4 GB | measured identical speed to Q4_K_M |
| Qwopus3.5-9B-Coder-DFlash-bf16.gguf | 2.6 GB | for requantizing |

## Measured (RTX 3070 8 GB, target Q3_K_M, temp 0, back to back, both fully on GPU)

| workload | baseline | DFlash | speedup | acceptance |
| --- | --- | --- | --- | --- |
| code editing (rename a field, echo the file) | 62 tok/s | 304 tok/s | 4.9x | 0.84 |
| fresh code generation | 58 tok/s | 145 tok/s | 2.5x | 0.34 |

The speedup tracks output predictability; editing existing code is the best case (mean draft length 13.7 of 15). Freeform prose drops to about 0.15 acceptance, still a net win. Both models plus buffers need about 6.5 GB free VRAM and a low fit margin (`-fitt 256`); if the target spills layers to CPU, speculation goes net-negative.

Also works on Apple Silicon via [dflash-mlx](https://github.com/bstnxbt/dflash-mlx): on an M3 Pro 18 GB, code editing goes 28 to 55 tok/s (1.95x) and fresh generation 28 to 42 tok/s (1.49x) with `--draft-quant w4 --block-tokens 8`. Setup scripts in the [GitHub repo](https://github.com/Gaurav-Gosain/qwopus-dflash).