---
license: apache-2.0
language:
  - en
  - zh
base_model:
  - TaichuAI/ZDTaichu5.0-9B
  - TaichuAI/ZDTaichu5.0-9B-DSpark
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - speculative-decoding
  - draft-model
  - dspark
  - dflash
  - zdtaichu5_0
  - qwen3.5
  - agent
---

[![🤗 Model Requests](https://img.shields.io/badge/%F0%9F%A4%97_Model_Requests-Submit_Here-f59e0b?style=for-the-badge&labelColor=1f2937)](https://abenzerps.com/request-a-model/)

# ZDTaichu5.0-9B DSpark GGUF

GGUF files for [ZDTaichu5.0-9B-DSpark](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B-DSpark), the official semi-autoregressive speculative decoding draft model developed by the Zi Dong Tai Chu team to accelerate [ZDTaichu5.0-9B](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B).

> [!NOTE]
> This is a **speculative decoding draft model** (D-Spark / DFlash), not a standalone general model. It must be paired with the primary [ZDTaichu5.0-9B GGUF](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF) model during inference to increase generation speeds by up to 60–85%.

## Benchmarks

![ZDTaichu5.0-9B benchmark results](assets/taichu-vs-closed-models.png)

*Benchmark results reported by TaichuAI for the original ZDTaichu5.0-9B foundation model. These figures are not measurements of this GGUF conversion.*

## GGUF files

| **Quantization** | **DSpark Draft File** | **Draft Size** | **Paired Base Model File** |
| :--- | :--- | ---: | :--- |
| **BF16** | [**ZDTaichu5.0-9B-DSpark-BF16.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-BF16.gguf) | 4.40 GB | [TaichuAI/ZDTaichu5.0-9B](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B) *(Original Weights)* |
| **Q8_0** | [**ZDTaichu5.0-9B-DSpark-Q8_0.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-Q8_0.gguf) | 2.34 GB | [ZDTaichu5.0-9B-Q8_0.gguf](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF/blob/main/ZDTaichu5.0-9B-Q8_0.gguf) (9.53 GB) |
| **Q6_K** | [**ZDTaichu5.0-9B-DSpark-Q6_K.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-Q6_K.gguf) | 1.81 GB | [ZDTaichu5.0-9B-Q6_K.gguf](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF/blob/main/ZDTaichu5.0-9B-Q6_K.gguf) (7.36 GB) |
| **Q5_K_M** | [**ZDTaichu5.0-9B-DSpark-Q5_K_M.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-Q5_K_M.gguf) | 1.67 GB | [ZDTaichu5.0-9B-Q5_K_M.gguf](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF/blob/main/ZDTaichu5.0-9B-Q5_K_M.gguf) (6.47 GB) |
| **Q4_K_M** | [**ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf) | 1.53 GB | [ZDTaichu5.0-9B-Q4_K_M.gguf](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF/blob/main/ZDTaichu5.0-9B-Q4_K_M.gguf) (5.63 GB) |
| **Q4_0** | [**ZDTaichu5.0-9B-DSpark-Q4_0.gguf**](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-DSpark-GGUF/blob/main/ZDTaichu5.0-9B-DSpark-Q4_0.gguf) | 1.51 GB | [ZDTaichu5.0-9B-Q4_0.gguf](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF/blob/main/ZDTaichu5.0-9B-Q4_0.gguf) (5.31 GB) |

### Draft model architecture

- **Framework:** DSpark (semi-autoregressive DFlash backbone + chained Markov head).
- **Target Layers:** Extracts auxiliary hidden states from target layers `[1, 8, 15, 22, 29]`.
- **Block Size:** 8 tokens drafted per forward pass (`dflash.block_size = 8`).
- **Target Model:** [ZDTaichu5.0-9B GGUF](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF).

## Usage

Use this draft model alongside the primary target model [abenzerps/ZDTaichu5.0-9B-GGUF](https://huggingface.co/abenzerps/ZDTaichu5.0-9B-GGUF) with `llama.cpp` (`llama-cli` or `llama-server`):

### Text generation with Speculative Decoding (`llama-cli`)

```bash
llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  -c 8192 --jinja \
  --temp 0.7 --top-p 0.95 \
  -p "Explain why reproducible builds matter."
```

### Multimodal vision understanding with Speculative Decoding (`llama-cli`)

```bash
llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  --image path/to/image.jpg \
  -p "Describe what is shown in this image in detail."
```

### OpenAI-compatible server with Speculative Decoding (`llama-server`)

```bash
llama-server \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  -c 8192 --jinja -fa on --host 0.0.0.0 --port 8080
```

Increase context `-c` up to 131,072 tokens (128K) when sufficient memory is available.

## Source and build

- Source draft model: [TaichuAI/ZDTaichu5.0-9B-DSpark](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B-DSpark)
- Target base model: [TaichuAI/ZDTaichu5.0-9B](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B)
- Conversion: [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) commit [7ceed8737fdb4eb09b4760e77bd12d38012de5a8](https://github.com/ggml-org/llama.cpp/commit/7ceed8737fdb4eb09b4760e77bd12d38012de5a8)
- License: [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
- Checksums: [SHA256SUMS.txt](SHA256SUMS.txt)