Instructions to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Anbeeld/Qwen3.8-27B-DFlash2-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Anbeeld/Qwen3.8-27B-DFlash2-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Anbeeld/Qwen3.8-27B-DFlash2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Anbeeld/Qwen3.8-27B-DFlash2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

SGLang

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Anbeeld/Qwen3.8-27B-DFlash2-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Anbeeld/Qwen3.8-27B-DFlash2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Anbeeld/Qwen3.8-27B-DFlash2-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Anbeeld/Qwen3.8-27B-DFlash2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Ollama:
```
ollama run hf.co/Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Docker Model Runner:
```
docker model run hf.co/Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M
```

Lemonade

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.8-27B-DFlash2-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Anbeeld/Qwen3.8-27B-DFlash2-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Anbeeld/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Anbeeld commited on 17 days ago

Commit

804ce82

verified ·

1 Parent(s): 50c22c4

Restore README image asset

Browse files

Files changed (1) hide show

README.md +285 -285

README.md CHANGED Viewed

@@ -1,285 +1,285 @@
----
-base_model: incoai/Qwen3.8-27B-DFlash2
-tags:
-- transformers
-- safetensors
-- qwen3
-- dflash2
-- speculative-decoding
-- block-diffusion
-- draft-model
-- sglang
-- vllm
-- text-generation
-- base_model:Qwen/Qwen3.8-27B
-- base_model:finetune:Qwen/Qwen3.8-27B
-- license:apache-2.0
-- text-generation-inference
-- region:us
----
-# Qwen 3.8 27B DFlash2 GGUF
-GGUF quantizations of [**Inco AI DFlash2 draft model**](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) for [**Qwen 3.8 27B**](https://huggingface.co/Qwen/Qwen3.8-27B).
-Use with [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp), a llama.cpp fork with advanced quantization features.
-## Benchmark results
-### Machine configuration
-- OS: Windows 11 Pro
-- CPU: AMD Ryzen 9 9950X 16-Core Processor
-- GPU: NVIDIA GeForce RTX 3090
-- Server CPU thread pool: 16 threads
-### Test setup
-- Runtime: [BeeLlama.cpp v0.4.4](https://github.com/Anbeeld/beellama.cpp), CUDA 13.1
-- Target model: [Qwen3.8-27B-UD-Q4_K_M](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)
-- 15 generations per each prompt + quant pair
-- DFlash2 setup: 7 draft tokens per block
-- Context: 8192 tokens
-- Batch size 4096, micro-batch size 1024
-- Temperature 1.0, top-p 0.95, top-k 20
-- Reasoning disabled
-### Task store module
-<details>
-<summary>Show prompt</summary>
-```text
-Write one complete Python 3 file using only the standard library.
-Return only Python code. Do not use markdown, comments, tests, examples, or explanatory text.
-Implement a deterministic Task store module with a compact, repetitive structure that is easy to predict.
-Required shape:
-- imports: dataclasses, datetime, typing
-- dataclass Task with fields id: int, title: str, status: str, created_at: str
-- class TaskStore with an internal dict[int, Task]
-- methods: add, get, rename, mark_done, reopen, delete, clear, list_all, list_open, list_done, count_open, count_done, titles, to_dicts, __len__, __contains__
-- add assigns increasing integer ids starting at 1
-- valid statuses are "open" and "done"
-- all list methods return tasks sorted by id
-- count_open and count_done use explicit loops
-- titles returns task titles sorted by task id
-- to_dicts returns deterministic dictionaries sorted by id
-- to_dicts includes id, title, status, and created_at keys for every task
-- raise ValueError for empty title or missing task id
-- use straightforward if statements and explicit loops
-- keep method bodies short and similar in style
-- no argparse, no JSON, no file IO, no unittest, no pytest
-- target about 110 to 132 lines of code
-- define __all__ = ["Task", "TaskStore"]
-- stop immediately after defining __all__
-```
-</details>
-| Quant type | Size (MB) | Mean length | Median tok/s | Mean tok/s | Median AR | Mean AR |
-|---|---:|---:|---:|---:|---:|---:|
-| Baseline | n/a | 897.93 | 42.222 | 42.237 | n/a | n/a |
-| Q2_K | 705.43 | 891.20 | 109.128 | 109.079 | 0.912 | 0.913 |
-| Q3_K_M | 916.70 | 891.20 | 108.330 | 108.747 | 0.916 | 0.915 |
-| Q4_K_M | 1,143.01 | 893.00 | 108.123 | 107.906 | 0.910 | 0.908 |
-| Q5_K_M | 1,359.93 | 899.87 | 107.408 | 107.332 | 0.906 | 0.907 |
-| Q6_K | 1,590.41 | 906.80 | 106.738 | 107.123 | 0.895 | 0.900 |
-| Q8_0 | 2,056.41 | 890.93 | 109.767 | 109.067 | 0.916 | 0.911 |
-| bf16 | 3,860.29 | 899.47 | 108.508 | 108.468 | 0.915 | 0.914 |
-### Key-value report module
-<details>
-<summary>Show prompt</summary>
-```text
-Write one complete Python 3 file using only the standard library.
-Return only Python code. Do not use markdown, comments, tests, examples, or explanatory text.
-Implement a deterministic KV report module with a compact, repetitive structure that is easy to predict.
-Required shape:
-- imports: dataclasses, typing
-- dataclass Row with fields key: str, value: str
-- class Report with an internal list[Row]
-- methods: add, set, get, delete, clear, keys, values, items, sorted_rows, render_lines, render_text, render_csv, filter_prefix, update_many, to_dict, copy, count_prefix, first_key, __len__, __contains__
-- add appends a new row and rejects duplicate keys
-- set updates an existing row or appends a new row
-- get returns the value for a key
-- delete removes a row by key
-- keys, values, and items preserve insertion order
-- sorted_rows returns rows sorted by key
-- render_lines returns strings formatted as "key: value"
-- render_text joins render_lines with newline characters
-- render_csv returns deterministic "key,value" lines with a header
-- filter_prefix returns a new Report containing keys that start with the prefix
-- update_many applies set for each key and value in a dictionary sorted by key
-- to_dict returns a deterministic dictionary sorted by key
-- copy returns a new Report with the same rows in the same order
-- count_prefix returns the number of keys that start with the prefix using an explicit loop
-- first_key returns the first key and raises ValueError when there are no rows
-- raise ValueError for empty keys, duplicate keys, or missing keys
-- use straightforward if statements and explicit loops
-- keep method bodies short and similar in style
-- no enum, no alignment modes, no markdown table, no textwrap, no itertools, no unittest, no pytest
-- target about 130 to 155 lines of code
-- define __all__ = ["Row", "Report"]
-- stop immediately after defining __all__
-```
-</details>
-| Quant type | Size (MB) | Mean length | Median tok/s | Mean tok/s | Median AR | Mean AR |
-|---|---:|---:|---:|---:|---:|---:|
-| Baseline | n/a | 1,026.60 | 41.865 | 41.829 | n/a | n/a |
-| Q2_K | 705.43 | 980.80 | 105.876 | 105.453 | 0.888 | 0.886 |
-| Q3_K_M | 916.70 | 1,007.53 | 105.543 | 105.158 | 0.885 | 0.883 |
-| Q4_K_M | 1,143.01 | 1,002.20 | 106.475 | 105.659 | 0.895 | 0.890 |
-| Q5_K_M | 1,359.93 | 1,003.00 | 104.483 | 104.082 | 0.886 | 0.883 |
-| Q6_K | 1,590.41 | 1,022.80 | 107.050 | 106.483 | 0.900 | 0.894 |
-| Q8_0 | 2,056.41 | 992.40 | 108.184 | 108.047 | 0.893 | 0.895 |
-| bf16 | 3,860.29 | 1,001.47 | 103.355 | 103.295 | 0.874 | 0.878 |
----
-# Qwen3.8-27B-DFlash2
-[Blog](https://inco.ai/blog/dflash2/) | [GitHub](https://github.com/z-lab/dflash)
-This repository contains the DFlash 2 draft model for
-[`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
-It is not a standalone language model: it runs inside a speculative
-decoding server and drafts tokens for the target model to verify. The checkpoint is also
-mirrored at [`z-lab/Qwen3.8-27B-DFlash2`](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2).
-DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts
-a whole block of tokens in a single pass and keeps the top candidates at
-every position. A lightweight selector then traces one coherent path through them.
-Two-tap dynamic convolutions in the backbone keep the draft from decaying
-toward the end of the block. Decoding is lossless: greedy output
-matches the target model exactly, and sampling preserves its distribution.
-<div align="center">
-  <img src="https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/resolve/main/assets/dflash2-figure.png" alt="DFlash 2: parallel block drafting with a candidate path selector" width="100%">
-</div>
-## Quick Start
-Serve with [SGLang](https://github.com/sgl-project/sglang):
-```bash
-pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
-python -m sglang.launch_server \
-  --model-path Qwen/Qwen3.8-27B \
-  --speculative-algorithm DFLASH \
-  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
-  --speculative-num-draft-tokens 8
-```
-Or with [vLLM](https://github.com/vllm-project/vllm):
-```bash
-pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"
-vllm serve Qwen/Qwen3.8-27B \
-  --speculative-config '{
-    "method": "dflash",
-    "model": "incoai/Qwen3.8-27B-DFlash2",
-    "num_speculative_tokens": 7
-  }'
-```
-See the [blog post](https://inco.ai/blog/dflash2/) for other engines and more details.
-## Evaluation
-- Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention
-- Speculation block size: 8 (7 draft tokens per verification step)
-- Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with `xhigh` reasoning effort
-- Maximum new tokens: 4096
-- Prompts: benchmark formatting from [`z-lab/dflash`](https://github.com/z-lab/dflash)
-We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP,
-a community DSpark drafter
-([`RadixArk/Qwen3.8-27B-DSpark`](https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark)),
-and DFlash 2. All speculative methods propose seven draft tokens per
-verification step.
-### Acceptance Length
-Acceptance length is the per-request mean of completion tokens divided by verification steps.
-Higher is better.
-| Task | MTP | DSpark | DFlash 2 |
-| :--- | ---: | ---: | ---: |
-| GSM8K | 5.02 | 4.36 | **5.46** |
-| MATH-500 | 4.72 | 3.92 | **5.28** |
-| HumanEval | 3.91 | 3.30 | **4.39** |
-| MBPP | 3.99 | 3.51 | **4.79** |
-| MT-Bench | 3.74 | 3.01 | **4.10** |
-### Throughput
-Throughput is total output tokens divided by end-to-end wall time.
-Each cell shows `output tok/s (speedup vs. autoregressive)`.
-#### Concurrency 1
-| Task | Autoregressive | MTP | DSpark | DFlash 2 |
-| :--- | ---: | ---: | ---: | ---: |
-| GSM8K | 68.9 | 178.5 (2.59×) | 185.3 (2.69×) | **236.1 (3.43×)** |
-| MATH-500 | 69.0 | 172.8 (2.51×) | 174.5 (2.53×) | **230.7 (3.34×)** |
-| HumanEval | 69.0 | 151.9 (2.20×) | 159.9 (2.32×) | **214.6 (3.11×)** |
-| MBPP | 69.0 | 153.1 (2.22×) | 163.3 (2.37×) | **226.9 (3.29×)** |
-| MT-Bench | 68.9 | 134.9 (1.96×) | 137.6 (2.00×) | **184.0 (2.67×)** |
-#### Concurrency 8
-| Task | Autoregressive | MTP | DSpark | DFlash 2 |
-| :--- | ---: | ---: | ---: | ---: |
-| GSM8K | 467.2 | 1,022.1 (2.19×) | 1,040.8 (2.23×) | **1,328.7 (2.84×)** |
-| MATH-500 | 480.0 | 1,023.5 (2.13×) | 1,025.8 (2.14×) | **1,368.3 (2.85×)** |
-| HumanEval | 483.4 | 934.2 (1.93×) | 956.5 (1.98×) | **1,291.5 (2.67×)** |
-| MBPP | 478.0 | 938.1 (1.96×) | 974.1 (2.04×) | **1,328.0 (2.78��)** |
-| MT-Bench | 480.5 | 835.2 (1.74×) | 802.3 (1.67×) | **1,090.2 (2.27×)** |
-#### Concurrency 32
-| Task | Autoregressive | MTP | DSpark | DFlash 2 |
-| :--- | ---: | ---: | ---: | ---: |
-| GSM8K | 1,329.8 | 1,381.1 (1.04×) | 1,506.5 (1.13×) | **1,922.5 (1.45×)** |
-| MATH-500 | 1,505.8 | 1,415.6 (0.94×) | 1,429.0 (0.95×) | **1,951.8 (1.30×)** |
-| HumanEval | 1,546.5 | 1,296.8 (0.84×) | 1,330.1 (0.86×) | **1,799.0 (1.16×)** |
-| MBPP | 1,507.7 | 1,314.9 (0.87×) | 1,361.3 (0.90×) | **1,886.8 (1.25×)** |
-| MT-Bench | 1,507.4 | 1,159.7 (0.77×) | 1,115.5 (0.74×) | **1,525.3 (1.01×)** |
-## Citation
-If you find DFlash 2 useful, please cite:
-```bibtex
-@misc{inco2026dflash2,
-  title  = {{DFlash 2: Keep Drafting Parallel}},
-  author = {{Inco AI}},
-  year   = {2026},
-  month  = {August},
-  url    = {https://inco.ai/blog/dflash2/}
-}
-```
-Please also cite the original DFlash paper:
-```bibtex
-@inproceedings{chen2026dflash,
-  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
-  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
-  booktitle = {International Conference on Machine Learning (ICML)},
-  year      = {2026}
-}
-```

+---
+base_model: incoai/Qwen3.8-27B-DFlash2
+tags:
+- transformers
+- safetensors
+- qwen3
+- dflash2
+- speculative-decoding
+- block-diffusion
+- draft-model
+- sglang
+- vllm
+- text-generation
+- base_model:Qwen/Qwen3.8-27B
+- base_model:finetune:Qwen/Qwen3.8-27B
+- license:apache-2.0
+- text-generation-inference
+- region:us
+---
+# Qwen 3.8 27B DFlash2 GGUF
+GGUF quantizations of [**Inco AI DFlash2 draft model**](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) for [**Qwen 3.8 27B**](https://huggingface.co/Qwen/Qwen3.8-27B).
+Use with [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp), a llama.cpp fork with advanced quantization features.
+## Benchmark results
+### Machine configuration
+- OS: Windows 11 Pro
+- CPU: AMD Ryzen 9 9950X 16-Core Processor
+- GPU: NVIDIA GeForce RTX 3090
+- Server CPU thread pool: 16 threads
+### Test setup
+- Runtime: [BeeLlama.cpp v0.4.4](https://github.com/Anbeeld/beellama.cpp), CUDA 13.1
+- Target model: [Qwen3.8-27B-UD-Q4_K_M](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)
+- 15 generations per each prompt + quant pair
+- DFlash2 setup: 7 draft tokens per block
+- Context: 8192 tokens
+- Batch size 4096, micro-batch size 1024
+- Temperature 1.0, top-p 0.95, top-k 20
+- Reasoning disabled
+### Task store module
+<details>
+<summary>Show prompt</summary>
+```text
+Write one complete Python 3 file using only the standard library.
+Return only Python code. Do not use markdown, comments, tests, examples, or explanatory text.
+Implement a deterministic Task store module with a compact, repetitive structure that is easy to predict.
+Required shape:
+- imports: dataclasses, datetime, typing
+- dataclass Task with fields id: int, title: str, status: str, created_at: str
+- class TaskStore with an internal dict[int, Task]
+- methods: add, get, rename, mark_done, reopen, delete, clear, list_all, list_open, list_done, count_open, count_done, titles, to_dicts, __len__, __contains__
+- add assigns increasing integer ids starting at 1
+- valid statuses are "open" and "done"
+- all list methods return tasks sorted by id
+- count_open and count_done use explicit loops
+- titles returns task titles sorted by task id
+- to_dicts returns deterministic dictionaries sorted by id
+- to_dicts includes id, title, status, and created_at keys for every task
+- raise ValueError for empty title or missing task id
+- use straightforward if statements and explicit loops
+- keep method bodies short and similar in style
+- no argparse, no JSON, no file IO, no unittest, no pytest
+- target about 110 to 132 lines of code
+- define __all__ = ["Task", "TaskStore"]
+- stop immediately after defining __all__
+```
+</details>
+| Quant type | Size (MB) | Mean length | Median tok/s | Mean tok/s | Median AR | Mean AR |
+|---|---:|---:|---:|---:|---:|---:|
+| Baseline | n/a | 897.93 | 42.222 | 42.237 | n/a | n/a |
+| Q2_K | 705.43 | 891.20 | 109.128 | 109.079 | 0.912 | 0.913 |
+| Q3_K_M | 916.70 | 891.20 | 108.330 | 108.747 | 0.916 | 0.915 |
+| Q4_K_M | 1,143.01 | 893.00 | 108.123 | 107.906 | 0.910 | 0.908 |
+| Q5_K_M | 1,359.93 | 899.87 | 107.408 | 107.332 | 0.906 | 0.907 |
+| Q6_K | 1,590.41 | 906.80 | 106.738 | 107.123 | 0.895 | 0.900 |
+| Q8_0 | 2,056.41 | 890.93 | 109.767 | 109.067 | 0.916 | 0.911 |
+| bf16 | 3,860.29 | 899.47 | 108.508 | 108.468 | 0.915 | 0.914 |
+### Key-value report module
+<details>
+<summary>Show prompt</summary>
+```text
+Write one complete Python 3 file using only the standard library.
+Return only Python code. Do not use markdown, comments, tests, examples, or explanatory text.
+Implement a deterministic KV report module with a compact, repetitive structure that is easy to predict.
+Required shape:
+- imports: dataclasses, typing
+- dataclass Row with fields key: str, value: str
+- class Report with an internal list[Row]
+- methods: add, set, get, delete, clear, keys, values, items, sorted_rows, render_lines, render_text, render_csv, filter_prefix, update_many, to_dict, copy, count_prefix, first_key, __len__, __contains__
+- add appends a new row and rejects duplicate keys
+- set updates an existing row or appends a new row
+- get returns the value for a key
+- delete removes a row by key
+- keys, values, and items preserve insertion order
+- sorted_rows returns rows sorted by key
+- render_lines returns strings formatted as "key: value"
+- render_text joins render_lines with newline characters
+- render_csv returns deterministic "key,value" lines with a header
+- filter_prefix returns a new Report containing keys that start with the prefix
+- update_many applies set for each key and value in a dictionary sorted by key
+- to_dict returns a deterministic dictionary sorted by key
+- copy returns a new Report with the same rows in the same order
+- count_prefix returns the number of keys that start with the prefix using an explicit loop
+- first_key returns the first key and raises ValueError when there are no rows
+- raise ValueError for empty keys, duplicate keys, or missing keys
+- use straightforward if statements and explicit loops
+- keep method bodies short and similar in style
+- no enum, no alignment modes, no markdown table, no textwrap, no itertools, no unittest, no pytest
+- target about 130 to 155 lines of code
+- define __all__ = ["Row", "Report"]
+- stop immediately after defining __all__
+```
+</details>
+| Quant type | Size (MB) | Mean length | Median tok/s | Mean tok/s | Median AR | Mean AR |
+|---|---:|---:|---:|---:|---:|---:|
+| Baseline | n/a | 1,026.60 | 41.865 | 41.829 | n/a | n/a |
+| Q2_K | 705.43 | 980.80 | 105.876 | 105.453 | 0.888 | 0.886 |
+| Q3_K_M | 916.70 | 1,007.53 | 105.543 | 105.158 | 0.885 | 0.883 |
+| Q4_K_M | 1,143.01 | 1,002.20 | 106.475 | 105.659 | 0.895 | 0.890 |
+| Q5_K_M | 1,359.93 | 1,003.00 | 104.483 | 104.082 | 0.886 | 0.883 |
+| Q6_K | 1,590.41 | 1,022.80 | 107.050 | 106.483 | 0.900 | 0.894 |
+| Q8_0 | 2,056.41 | 992.40 | 108.184 | 108.047 | 0.893 | 0.895 |
+| bf16 | 3,860.29 | 1,001.47 | 103.355 | 103.295 | 0.874 | 0.878 |
+---
+# Qwen3.8-27B-DFlash2
+[Blog](https://inco.ai/blog/dflash2/) | [GitHub](https://github.com/z-lab/dflash)
+This repository contains the DFlash 2 draft model for
+[`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
+It is not a standalone language model: it runs inside a speculative
+decoding server and drafts tokens for the target model to verify. The checkpoint is also
+mirrored at [`z-lab/Qwen3.8-27B-DFlash2`](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2).
+DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts
+a whole block of tokens in a single pass and keeps the top candidates at
+every position. A lightweight selector then traces one coherent path through them.
+Two-tap dynamic convolutions in the backbone keep the draft from decaying
+toward the end of the block. Decoding is lossless: greedy output
+matches the target model exactly, and sampling preserves its distribution.
+<div align="center">
+  <img src="assets/dflash2-figure.png" alt="DFlash 2: parallel block drafting with a candidate path selector" width="100%">
+</div>
+## Quick Start
+Serve with [SGLang](https://github.com/sgl-project/sglang):
+```bash
+pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3.8-27B \
+  --speculative-algorithm DFLASH \
+  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
+  --speculative-num-draft-tokens 8
+```
+Or with [vLLM](https://github.com/vllm-project/vllm):
+```bash
+pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"
+vllm serve Qwen/Qwen3.8-27B \
+  --speculative-config '{
+    "method": "dflash",
+    "model": "incoai/Qwen3.8-27B-DFlash2",
+    "num_speculative_tokens": 7
+  }'
+```
+See the [blog post](https://inco.ai/blog/dflash2/) for other engines and more details.
+## Evaluation
+- Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention
+- Speculation block size: 8 (7 draft tokens per verification step)
+- Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with `xhigh` reasoning effort
+- Maximum new tokens: 4096
+- Prompts: benchmark formatting from [`z-lab/dflash`](https://github.com/z-lab/dflash)
+We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP,
+a community DSpark drafter
+([`RadixArk/Qwen3.8-27B-DSpark`](https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark)),
+and DFlash 2. All speculative methods propose seven draft tokens per
+verification step.
+### Acceptance Length
+Acceptance length is the per-request mean of completion tokens divided by verification steps.
+Higher is better.
+| Task | MTP | DSpark | DFlash 2 |
+| :--- | ---: | ---: | ---: |
+| GSM8K | 5.02 | 4.36 | **5.46** |
+| MATH-500 | 4.72 | 3.92 | **5.28** |
+| HumanEval | 3.91 | 3.30 | **4.39** |
+| MBPP | 3.99 | 3.51 | **4.79** |
+| MT-Bench | 3.74 | 3.01 | **4.10** |
+### Throughput
+Throughput is total output tokens divided by end-to-end wall time.
+Each cell shows `output tok/s (speedup vs. autoregressive)`.
+#### Concurrency 1
+| Task | Autoregressive | MTP | DSpark | DFlash 2 |
+| :--- | ---: | ---: | ---: | ---: |
+| GSM8K | 68.9 | 178.5 (2.59×) | 185.3 (2.69×) | **236.1 (3.43×)** |
+| MATH-500 | 69.0 | 172.8 (2.51×) | 174.5 (2.53×) | **230.7 (3.34×)** |
+| HumanEval | 69.0 | 151.9 (2.20×) | 159.9 (2.32×) | **214.6 (3.11×)** |
+| MBPP | 69.0 | 153.1 (2.22×) | 163.3 (2.37×) | **226.9 (3.29×)** |
+| MT-Bench | 68.9 | 134.9 (1.96×) | 137.6 (2.00×) | **184.0 (2.67×)** |
+#### Concurrency 8
+| Task | Autoregressive | MTP | DSpark | DFlash 2 |
+| :--- | ---: | ---: | ---: | ---: |
+| GSM8K | 467.2 | 1,022.1 (2.19×) | 1,040.8 (2.23×) | **1,328.7 (2.84×)** |
+| MATH-500 | 480.0 | 1,023.5 (2.13×) | 1,025.8 (2.14×) | **1,368.3 (2.85×)** |
+| HumanEval | 483.4 | 934.2 (1.93×) | 956.5 (1.98×) | **1,291.5 (2.67×)** |
+| MBPP | 478.0 | 938.1 (1.96×) | 974.1 (2.04×) | **1,328.0 (2.78×)** |
+| MT-Bench | 480.5 | 835.2 (1.74×) | 802.3 (1.67×) | **1,090.2 (2.27×)** |
+#### Concurrency 32
+| Task | Autoregressive | MTP | DSpark | DFlash 2 |
+| :--- | ---: | ---: | ---: | ---: |
+| GSM8K | 1,329.8 | 1,381.1 (1.04×) | 1,506.5 (1.13×) | **1,922.5 (1.45×)** |
+| MATH-500 | 1,505.8 | 1,415.6 (0.94×) | 1,429.0 (0.95×) | **1,951.8 (1.30×)** |
+| HumanEval | 1,546.5 | 1,296.8 (0.84×) | 1,330.1 (0.86×) | **1,799.0 (1.16×)** |
+| MBPP | 1,507.7 | 1,314.9 (0.87×) | 1,361.3 (0.90×) | **1,886.8 (1.25×)** |
+| MT-Bench | 1,507.4 | 1,159.7 (0.77×) | 1,115.5 (0.74×) | **1,525.3 (1.01×)** |
+## Citation
+If you find DFlash 2 useful, please cite:
+```bibtex
+@misc{inco2026dflash2,
+  title  = {{DFlash 2: Keep Drafting Parallel}},
+  author = {{Inco AI}},
+  year   = {2026},
+  month  = {August},
+  url    = {https://inco.ai/blog/dflash2/}
+}
+```
+Please also cite the original DFlash paper:
+```bibtex
+@inproceedings{chen2026dflash,
+  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
+  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
+  booktitle = {International Conference on Machine Learning (ICML)},
+  year      = {2026}
+}
+```