Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3.8-27b-opus-distill.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3.8-27b-opus-distill.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3.8-27b-opus-distill.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3.8-27b-opus-distill.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3.8-27b-opus-distill.f16.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,134 @@
+---
+license: apache-2.0
+datasets:
+  - lordx64/reasoning-distill-claude-opus-4-7-max
+  - Jackrong/Claude-opus-4.7-TraceInversion-5000x
+  - Jackrong/Claude-opus-4.6-TraceInversion-9000x
+language:
+  - en
+base_model: Qwen/Qwen3.8-27B
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+  - gguf
+  - llama.cpp
+  - unsloth
+  - lora
+  - reasoning
+  - chain-of-thought
+  - distillation
+  - claude
+  - claude-opus
+  - qwen
+  - qwen3.8
+  - dense
+  - mtp
+  - speculative-decoding
+---
+
+# Qwen3.8-27B-Claude-Opus-Reasoning-Distilled-GGUF
+
+![License](https://img.shields.io/badge/license-apache--2.0-blue)
+![Base Model](https://img.shields.io/badge/base-Qwen3.8--27B-orange)
+![Method](https://img.shields.io/badge/method-LoRA%20%2F%20Unsloth-purple)
+![Teacher](https://img.shields.io/badge/distilled%20from-Claude%20Opus-black)
+![MTP](https://img.shields.io/badge/MTP-verified%20working-brightgreen)
+
+GGUF quantizations of [rico03/Qwen3.8-27B-Claude-Opus-Reasoning-Distilled](https://huggingface.co/rico03/Qwen3.8-27B-Claude-Opus-Reasoning-Distilled) — a LoRA fine-tune of **Qwen/Qwen3.8-27B** distilled on reasoning traces from **Claude Opus 4.6/4.7**, merged to 16-bit and quantized for local inference with llama.cpp / Ollama / LM Studio.
+
+**MTP (Multi-Token Prediction) head verified working for self-speculative decoding — see below, ~1.6× speedup.**
+
+## 🧠 What this is
+
+Qwen3.8-27B is natively vision-language with a hybrid attention stack (16/64 full-attention layers, 48/64 Gated DeltaNet/linear-attention layers) plus a Multi-Token Prediction head. This fine-tune targets **only the text reasoning path**; the vision tower is present in the merge (unmodified base weights) but was not targeted by LoRA.
+
+- **Base model:** [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) (dense, 27B, hybrid linear+full attention, MTP head)
+- **Method:** LoRA (r=32, alpha=32) via [Unsloth](https://github.com/unslothai/unsloth), merged to 16-bit, converted with [llama.cpp](https://github.com/ggml-org/llama.cpp) (latest build — required for hybrid Gated DeltaNet operator support)
+- **Training data:** ~21.5K examples combining real Opus extended-thinking traces and reconstructed-reasoning traces over genuine Opus outputs (see Dataset Composition below)
+
+## 📊 Training Details
+
+| | |
+|---|---|
+| Steps trained | 150 (partial run — pipeline validation, not full convergence) |
+| Epochs | 0.126 (~12.6% of one epoch, ~2,700 of 21,490 examples) |
+| Effective batch size | 18 (per-device 6 × grad accum 3) |
+| Final train loss (avg) | 0.728 |
+| Sequence length | 8192 (99.7% of dataset uncut; 0.3% longest examples excluded rather than truncated) |
+| Trainable params (LoRA) | 233,455,616 (0.85% of 27.6B) |
+
+> **Status: pipeline-validation run, not a fully converged model.** 150 steps at batch 18 covers ~12.6% of one epoch — enough to confirm the training pipeline works end-to-end (LoRA on hybrid attention, masking, chat template, checkpointing, merge, GGUF conversion, MTP preservation), not enough for the model to have generalized across the full dataset. A full run (1+ epoch, ~1,194+ steps) is needed before this should be treated as a finished distillation.
+
+## 📚 Dataset Composition
+
+| Source | Examples | What it is |
+|---|---:|---|
+| [lordx64/reasoning-distill-claude-opus-4-7-max](https://huggingface.co/datasets/lordx64/reasoning-distill-claude-opus-4-7-max) | 8,124 | Genuine Claude Opus 4.7 extended-thinking traces (verified `model`/`usage` fields per row) |
+| [Jackrong/Claude-opus-4.7-TraceInversion-5000x](https://huggingface.co/datasets/Jackrong/Claude-opus-4.7-TraceInversion-5000x) | 4,800 | Real Opus 4.7 outputs (`teacher_model` field), reasoning trace reconstructed post-hoc by a smaller "trace inversion" model |
+| [Jackrong/Claude-opus-4.6-TraceInversion-9000x](https://huggingface.co/datasets/Jackrong/Claude-opus-4.6-TraceInversion-9000x) | 8,700 | Same trace-inversion method, Opus 4.6 |
+
+**Transparency note:** ~62% of the dataset has a *genuine* final answer from Opus but a *reconstructed* reasoning trace, not Opus's actual internal thinking. The lordx64 subset (~38%) is the closest to authentic Opus thinking style. Full details on the [base LoRA repo](https://huggingface.co/rico03/Qwen3.8-27B-Claude-Opus-Reasoning-Distilled).
+
+## ⚡ MTP / Speculative Decoding
+
+The Multi-Token Prediction head from the base model (`blk.64.nextn.*`) survives the LoRA merge and GGUF conversion intact and is functional for self-speculative decoding — verified on Q8_0:
+
+| Mode | Generation speed (H100 NVL) |
+|---|---:|
+| Standard decoding | 64.6 tok/s |
+| `--spec-type draft-mtp` | **104.2 tok/s** (~1.6× speedup) |
+
+```bash
+llama-cli -m qwen3.8-27b-opus-distill.Q8_0.gguf -p "your prompt" -ngl 99 --spec-type draft-mtp
+```
+
+No separate draft model needed — the MTP head inside this same GGUF file acts as the draft. Output quality is unaffected (MTP proposes candidate tokens that the main model verifies; it doesn't change what gets generated, only how fast). Requires a recent llama.cpp build with `--spec-type draft-mtp` support.
+
+## 📦 Available Quantizations
+
+| File | Size | Notes |
+|---|---:|---|
+| `qwen3.8-27b-opus-distill.f16.gguf` | 51 GB | Full precision, source for further quantization |
+| `qwen3.8-27b-opus-distill.Q8_0.gguf` | 28 GB | Near-lossless, recommended if VRAM allows |
+| `qwen3.8-27b-opus-distill.Q6_K.gguf` | 21 GB | Good quality/size trade-off |
+| `qwen3.8-27b-opus-distill.Q5_K_M.gguf` | 19 GB | |
+| `qwen3.8-27b-opus-distill.Q4_K_M.gguf` | 16 GB | Practical for single consumer GPU (24GB card) |
+
+All quants verified to generate coherent output (Q8_0 sanity-tested with a technical prompt before upload). MTP speculative decoding available on all quants sharing the same architecture.
+
+**⚠️ Requires a recent llama.cpp build.** This model uses Qwen3.8's hybrid Gated DeltaNet + Gated Attention architecture — older llama.cpp builds will not load it correctly. Build from source (`main` branch) if your package manager's version is stale.
+
+## 🚀 Usage
+
+### llama.cpp
+
+```bash
+./llama-cli -m qwen3.8-27b-opus-distill.Q8_0.gguf -p "Explain the difference between TCP and UDP." -ngl 99 --temp 1.0 --top-p 0.95 --top-k 20
+```
+
+With MTP speculative decoding (faster, same output distribution):
+
+```bash
+./llama-cli -m qwen3.8-27b-opus-distill.Q8_0.gguf -p "Explain the difference between TCP and UDP." -ngl 99 --temp 1.0 --top-p 0.95 --top-k 20 --spec-type draft-mtp
+```
+
+### Ollama / LM Studio
+
+Download the `.gguf` file matching your VRAM budget from the Files tab and point Ollama/LM Studio at it directly (Modelfile/import instructions per their respective docs).
+
+Thinking is on by default (`<think>...</think>` block before the final answer), matching the base Qwen3.8 chat template.
+
+## ⚠️ Known Limitations
+
+- **This checkpoint is a pipeline-validation run (150 steps, ~12.6% of one epoch), not a fully converged fine-tune.** Expect it to show the target `<think>` format and some stylistic shift, but not robust generalization across task domains.
+- Dataset partially composed of reconstructed (not captured) reasoning traces — see Dataset Composition above
+- Text-only fine-tune; vision tower weights are present in the merge (unmodified base) but untested/unused in this GGUF export
+- Requires latest llama.cpp for hybrid attention + MTP support (see note above)
+
+## 🙏 Acknowledgments
+
+Training methodology based on the [Jackrong fine-tuning guide](https://github.com/Jackrong). Thanks to lordx64 and Jackrong for the source reasoning datasets, and the llama.cpp team for hybrid-architecture support.
+
+---
+
+Maintained by rico03

qwen3.8-27b-opus-distill.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d88819a7bbbd1ee6cee2169eec16e8cc51de759cb6aff2f621e60564ec9a7ac6
+size 16810714336

qwen3.8-27b-opus-distill.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41c5bdc6719eb2f49fc28dffed471c57135de976774131045c1b57c1dbed382c
+size 19535701216

qwen3.8-27b-opus-distill.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f20a86da392455d4e41eb1a4049750fc7eab234d96483866047fb84c7ecbef16
+size 22430999776

qwen3.8-27b-opus-distill.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09379075cf890ebbef92886f40b58ebd0969422b7c32a4e764bf063a10c0dbea
+size 29047084256

qwen3.8-27b-opus-distill.f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb7e0fd4e23c97aea5199a5784e60875a8c8be07865217d265743eab6ed96a3b
+size 54657733856