Add limitations, detailed model info, and highlights

README.md

@@ -15,35 +15,96 @@ pipeline_tag: text-generation
 
 # Qwen3.5-9B-Claude-distill
 
-A distilled model fine-tuned from Qwen3.5-9B using Claude-generated reasoning traces. This model inherits structured reasoning patterns through full-parameter supervised fine-tuning.
+> A powerful reasoning model distilled from Claude Opus 4.6's chain-of-thought traces with enhanced capacity.
+
+## Introduction
+
+**Qwen3.5-9B-Claude-distill** is a distilled model fine-tuned from Qwen3.5-9B using Claude-generated reasoning traces. Through full-parameter supervised fine-tuning, this model inherits structured reasoning patterns from Claude Opus 4.6, enabling it to:
+
+- Break down complex problems into step-by-step reasoning
+- Follow structured thinking patterns within `沉浸在思考中...` blocks
+- Provide well-reasoned answers with transparent thought processes
+- Handle more complex reasoning chains than smaller variants
+
+### Key Highlights
+
+- **Full-parameter fine-tuning**: Unlike LoRA-based approaches, this model was trained with full parameter updates for deeper knowledge transfer
+- **Chain-of-thought distillation**: Trained on extended reasoning traces that expose the model to structured problem-solving patterns
+- **Balanced size-performance**: 9B parameters offer a sweet spot between capability and efficiency
 
 ## Model Details
 
-- **Base Model**: Qwen/Qwen3.5-9B
-- **Framework**: Swift 4.0.2
-- **Training Method**: Full-parameter fine-tuning (not LoRA)
-- **Training Samples**: ~11,800
+| Property | Value |
+|----------|-------|
+| Base Model | Qwen/Qwen3.5-9B |
+| Framework | Swift 4.0.2 |
+| Training Method | Full-parameter fine-tuning |
+| Training Samples | ~11,800 |
+| Context Length | 4096 tokens |
+| Precision | BF16 |
+
+### Training Configuration
+
+- Optimizer: AdamW (fused)
+- Learning Rate: 7e-6
+- Scheduler: Cosine with 5% warmup
+- DeepSpeed: ZeRO-2
 
 ## Training Data
 
-The model was trained on a curated collection of Claude-generated reasoning datasets:
+The model was trained on a curated collection of Claude Opus 4.6 generated reasoning datasets:
+
+| Dataset | Samples | Description |
+|---------|---------|-------------|
+| [claude-opus-4.6-10000x](https://huggingface.co/datasets/dalisoft/claude-opus-4.6-10000x) | ~10,000 | High-fidelity reasoning traces for mathematical accuracy and structured logical deduction |
+| [claude-opus-4.6-high-reasoning-700x](https://huggingface.co/datasets/dalisoft/claude-opus-4.6-high-reasoning-700x) | ~700 | Coding and creative writing with adaptive reasoning effort |
+| [Claude-Opus-4.6-Reasoning-927x](https://huggingface.co/datasets/dalisoft/Claude-Opus-4.6-Reasoning-927x) | ~927 | Tool calling, bullshit detection, and multi-turn reasoning |
+| [Claude-Sonnet-X-Opus-4.6-Reasoning-small-500](https://huggingface.co/datasets/dalisoft/Claude-Sonnet-X-Opus-4.6-Reasoning-small-500) | ~500 | Natural human-written prompts from Reddit and Stack Overflow |
 
-| Dataset | Description |
-|---------|-------------|
-| [claude-opus-4.6-10000x](https://huggingface.co/datasets/dalisoft/claude-opus-4.6-10000x) | High-fidelity reasoning traces for mathematical accuracy and structured logical deduction |
-| [claude-opus-4.6-high-reasoning-700x](https://huggingface.co/datasets/dalisoft/claude-opus-4.6-high-reasoning-700x) | Coding and creative writing with adaptive reasoning effort |
-| [Claude-Opus-4.6-Reasoning-927x](https://huggingface.co/datasets/dalisoft/Claude-Opus-4.6-Reasoning-927x) | Tool calling, bullshit detection, and multi-turn reasoning |
-| [Claude-Sonnet-X-Opus-4.6-Reasoning-small-500](https://huggingface.co/datasets/dalisoft/Claude-Sonnet-X-Opus-4.6-Reasoning-small-500) | Natural human-written prompts from Reddit and Stack Overflow |
+**Total**: ~12,100 high-quality reasoning samples (after deduplication and cleaning: ~11,800)
 
 ## Usage
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model = AutoModelForCausalLM.from_pretrained("Kassadin88/Qwen3.5-9B-Claude-distill", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    "Kassadin88/Qwen3.5-9B-Claude-distill",
+    trust_remote_code=True,
+    torch_dtype="auto"
+)
 tokenizer = AutoTokenizer.from_pretrained("Kassadin88/Qwen3.5-9B-Claude-distill")
+
+# Chat format
+messages = [
+    {"role": "system", "content": "You are a helpful AI assistant."},
+    {"role": "user", "content": "Solve: If 3x + 7 = 22, what is x?"}
+]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
+## Limitations
+
+- **Hallucination Risk**: While reasoning capabilities are enhanced, the model may still generate incorrect facts or hallucinate during the thinking process. External facts mentioned in reasoning traces should be verified independently.
+
+- **Training Data Bias**: The model inherits biases from the training data, which primarily consists of English content with some Chinese samples. Performance on other languages may be limited.
+
+- **Not a Replacement for Verification**: The structured reasoning outputs should not be treated as guaranteed correct answers. Critical applications require human verification.
+
+- **Limited Domain Coverage**: Training data focuses on math, logic, coding, and general problem-solving. Specialized domains (medical, legal, financial) are not well-represented.
+
+## Intended Use Cases
+
+- Offline analysis and reasoning tasks
+- Mathematical problem solving
+- Code generation and debugging
+- Logical deduction and planning
+- Educational applications where transparent reasoning is valuable
+- Complex multi-step reasoning tasks
+
 ## License
 
 Apache 2.0
@@ -52,4 +113,5 @@ Apache 2.0
 
 - [Swift](https://github.com/modelscope/swift) - The training framework
 - [Qwen Team](https://github.com/QwenLM) - The base model
-- Dataset creators: dalisoft and the open-source community
+- [dalisoft](https://huggingface.co/dalisoft) - Dataset creator
+- Open-source community for the reasoning datasets