Rustamshry commited on
Commit
96cfbbc
·
verified ·
1 Parent(s): 48e74ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -16
README.md CHANGED
@@ -12,19 +12,43 @@ base_model:
12
  pipeline_tag: image-text-to-text
13
  ---
14
 
15
- # Qwen3.8-max-Reasoning-Distilled-GGUF : GGUF
16
-
17
- This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
18
-
19
- **Example usage**:
20
- - For text only LLMs: `llama-cli -hf khazarai/Qwen3.8-max-Reasoning-Distilled-GGUF --jinja`
21
- - For multimodal models: `llama-mtmd-cli -hf khazarai/Qwen3.8-max-Reasoning-Distilled-GGUF --jinja`
22
-
23
- ## Available Model files:
24
- - `Qwen3.5-0.8B.BF16.gguf`
25
- - `Qwen3.5-0.8B.Q8_0.gguf`
26
- - `Qwen3.5-0.8B.Q6_K.gguf`
27
- - `Qwen3.5-0.8B.Q4_K_M.gguf`
28
- - `Qwen3.5-0.8B.F16-mmproj.gguf`
29
- This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
30
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  pipeline_tag: image-text-to-text
13
  ---
14
 
15
+ # Qwen3.8-max-Reasoning-Distilled
16
+
17
+ **Qwen3.8-max-Reasoning-Distilled** is a compact, high-efficiency language model fine-tuned using Knowledge Distillation from Qwen3.8-max.
18
+
19
+ It specifically addresses the common issue of overthinking loops and repetitive hesitations found in smaller raw reasoning models. By learning directly from high-quality reasoning traces, this model delivers concise, structured, and decisive Chain-of-Thought (CoT) outputs without wasting tokens on circular verification.
20
+
21
+ ## Key Features
22
+
23
+ - Streamlined Chain-of-Thought (CoT): Replaces verbose, rambling reasoning paths with direct, step-by-step logic.
24
+
25
+ - Elimination of Overthinking: Suppresses repetitive inner monologue loops (e.g., "Wait, let me re-verify...") that consume unnecessary tokens.
26
+
27
+ - Higher Inference Efficiency: Drastically reduces total generated token count while maintaining or improving final answer accuracy.
28
+
29
+ - Distilled Logic from Qwen3.8-max: Captures the complex evaluation heuristics of the teacher model into a smaller, faster student architecture.
30
+
31
+ ## Reasoning Quality Comparison
32
+
33
+ Below is a comparison highlighting how distillation improves reasoning structure and eliminates repetitive loops:
34
+
35
+ | **Feature** | **Qwen3.5-0.8B** | **Qwen3.8-max-Reasoning-Distilled** |
36
+ |---|---|---|
37
+ | **Logic Flow** | Frequent self-doubt, circular checks, hesitation | Linear, structured, and hypothesis-driven |
38
+ | **Token Economy** | High token overhead spent on repetitive verification | Low token overhead with concise step evaluation |
39
+ | **Option Analysis** | Reiterates choices multiple times without deciding | Evaluates each option once with clear justification |
40
+ | **Decision Speed** | Slow convergence to final answer | Fast, decisive output generation |
41
+
42
+ ## Example Trace
43
+
44
+ **Task**: Identify the beneficial effect of carpooling from multiple choice options.
45
+
46
+ - **Qwen3.5-0.8B**: "Evaluate option D... wait, let me check option E... actually, let me re-evaluate D... wait, is there a trick? Let me double-check..." (Overthinking Loop)
47
+ - **Qwen3.8-max-Reasoning-Distilled**: "Systematically evaluates choices A through H in a single pass $\rightarrow$ identifies option D as the primary environmental benefit $\rightarrow$ concludes decisively." (Clean & Direct)
48
+
49
+ ## Limitations
50
+
51
+ While the model minimizes overthinking, extremely complex multi-step mathematical problems may still require prompting for explicit scratchpad steps.
52
+ Always verify outputs when using the model for domain-critical tasks (e.g., medical, legal, or financial decisions).
53
+
54
+