README.md

---
library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE
pipeline_tag: text-generation
base_model: Qwen/Qwen3.5-9B
tags:
- qwen3.5
- claude-distill
language:
- en
- zh
---

# Qwen3.5-9B Claude-Distill

A fine-tuned version of [Qwen/Qwen3.5-9B](Qwen/Qwen3.5-9B_URL) through knowledge distillation from Claude. This model is trained with **full parameter fine-tuning** on curated Claude reasoning traces.

## Model Highlights

- **Claude-Distilled Reasoning**: Trained on high-quality chain-of-thought reasoning traces distilled from Claude Opus
- **Multi-Domain Coverage**: Math, logic, coding, creative writing, STEM, and multi-turn reasoning
- **Dense Architecture**: Based on Qwen/Qwen3.5-9B with 9B parameters
- **Multimodal Capable**: Inherits vision-language capabilities from Qwen3.5

## Model Description

| Property | Value |
|----------|-------|
| **Base Model** | Qwen/Qwen3.5-9B |
| **Model Type** | Causal Language Model with Vision Encoder |
| **Parameters** | 9B |
| **Languages** | English, Chinese |
| **License** | Apache 2.0 |
| **Developer** | [Kassadin88](https://huggingface.co/Kassadin88) |

## Training Data

Distilled from Claude on the following datasets:

| Dataset | Samples | Description |
|---------|---------|-------------|
| [Claude Opus 4.5 High Reasoning](https://huggingface.co/datasets/dalisoft/claude-4.5-opus-high-reasoning-250x) | 250 | High reasoning depth samples |
| [Claude Opus 4.6 Reasoning](https://huggingface.co/datasets/V3N0M/Jenna-Opus-4.6) | 9,633 | Math, logic puzzles, multi-step instructions with CoT |
| [Claude Opus 4.6 High Reasoning](https://huggingface.co/datasets/dalisoft/claude-opus-4.6-high-reasoning-700x) | 757 | Coding and creative writing with adaptive reasoning |
| [Claude Opus 4.6 Extended Reasoning](https://huggingface.co/datasets/Vezora/Claude-Opus-4.6-Reasoning-500x) | 500 | Extended reasoning across STEM and practical domains |
| [Claude Opus 4.6 Extended Reasoning 887x](https://huggingface.co/datasets/Vezora/Claude-Opus-4.6-Reasoning-887x) | 887 | Tool calling, bullshit detection, multi-turn traces |
| [Claude Sonnet & Opus 4.6 Reasoning](https://huggingface.co/datasets/riddlemeasured/Claude-Sonnet-X-Opus-4.6-Reasoning-small-500) | 524 | Natural human-written prompts from Reddit & Stack Overflow |
| [Opus 4.6 Reasoning Filtered](https://huggingface.co/datasets/nickexyi/Opus-4.6-Reasoning-3000x-filtered) | 2,326 | Filtered reasoning traces (refusals removed) |

**Total: ~14.9K samples**

### Data Composition

| Domain | Percentage | Description |
|--------|------------|-------------|
| **Math & Logic** | ~40% | Multi-step problem solving with chain-of-thought |
| **Coding** | ~25% | Code generation, debugging, and algorithm design |
| **STEM** | ~15% | Science, engineering, and extended reasoning |
| **Creative Writing** | ~10% | Adaptive reasoning for creative tasks |
| **Multi-turn / Tool Use** | ~10% | Tool calling, clarification, and dialogue |

## Benchmark Results

![Benchmark Results](https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/Qwen3.5/Figures/qwen3.5_small_size_score.png)

For detailed benchmark results and model architecture, please refer to the original [Qwen/Qwen3.5-9B](Qwen/Qwen3.5-9B_URL) model card.

## Quickstart

For full usage guide, please refer to the original [Qwen/Qwen3.5-9B](Qwen/Qwen3.5-9B_URL) model card.

### Using with vLLM

```bash
vllm serve Kassadin88/Qwen3.5-9B-Claude-distill \
    --port 8000 \
    --tensor-parallel-size 2 \
    --max-model-len 32768 \
    --trust-remote-code \
    --reasoning-parser qwen3
```

### Using with SGLang

```bash
python -m sglang.launch_server \
    --model-path Kassadin88/Qwen3.5-9B-Claude-distill \
    --port 8000 \
    --tp-size 2 \
    --mem-fraction-static 0.8 \
    --context-length 32768 \
    --reasoning-parser qwen3
```

### Using with Hugging Face Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Kassadin88/Qwen3.5-9B-Claude-distill"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {"role": "user", "content": "Hello, how are you?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
```

## Usage Tips

### For Reasoning Tasks

```python
messages = [
    {"role": "user", "content": "Solve step by step: What is the sum of all prime numbers less than 100?"}
]
# Model will use chain-of-thought reasoning from Claude distillation
```

### For Coding Tasks

```python
messages = [
    {"role": "user", "content": "Implement a binary search tree with insert, delete, and find operations in Python."}
]
# Model benefits from Claude's coding reasoning traces
```

### Enabling / Disabling Thinking

```python
# Enable thinking mode (recommended for reasoning tasks)
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)

# Disable thinking mode (for simple tasks, faster inference)
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
```

## Limitations

- This model is distilled from Claude and may inherit biases from the training data
- The distillation dataset is relatively small (~14.9K samples), which may limit generalization
- Should not be used for medical, legal, or financial advice without verification
- The model's reasoning capabilities are constrained by the quality and diversity of the distillation data

## Citation

```bibtex
@misc{qwen3.5-9b-claude-distill,
    author = {Kassadin88},
    title = {Qwen3.5-9B Claude-Distill: A Claude-Distilled Fine-Tuned Model},
    year = {2026},
    publisher = {HuggingFace},
    url = {https://huggingface.co/Kassadin88/Qwen3.5-9B-Claude-distill}
}
```

## Acknowledgments

- **Base Model**: [Qwen Team](https://github.com/QwenLM/Qwen3) for Qwen3.5
- **Training Data**: Various Claude Opus reasoning datasets on HuggingFace
- **Training Framework**: DeepSpeed

---

**Note:** This model is intended for research and educational purposes. Please use responsibly.