Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,136 @@
+---
+library_name: llama.cpp
+base_model:
+  - Jackrong/Qwopus3.6-27B-Coder-MTP
+base_model_relation: quantized
+license: apache-2.0
+language:
+  - en
+  - zh
+  - es
+  - ru
+  - ja
+pipeline_tag: image-text-to-text
+tags:
+  - gguf
+  - llama.cpp
+  - nvfp4
+  - blackwell
+  - vision
+  - multimodal
+  - mtp
+  - speculative-decoding
+  - qwen3_6
+  - reasoning
+  - coder
+  - agent
+  - tool-use
+---
+
+# Qwopus3.6-27B-Coder-NVFP4-MTP-GGUF
+
+NVFP4 (Blackwell native FP4) quantized GGUF of [Jackrong/Qwopus3.6-27B-Coder-MTP](https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF) for llama.cpp with Multi-Token Prediction (MTP) speculative decoding support.
+
+## Quantization Details
+
+| Attribute | Value |
+|-----------|-------|
+| **Source** | Q8_0 GGUF (28 GB, 8.50 BPW) |
+| **Output** | Mixed-precision NVFP4 (15 GB, 4.60 BPW) |
+| **Size reduction** | 46% (28 GB → 15 GB) |
+| **NVFP4 tensors** | 311 (attn_q, attn_k, attn_v, attn_qkv, attn_output, ffn_down, ffn_gate, ffn_up) |
+| **Q4_K tensors** | 194 (attn_gate, ssm_alpha, ssm_beta, ssm_out, nextn.eh_proj, token_embd) |
+| **Q4_K_S tensors** | 1 (output.weight) |
+| **F32 tensors** | 360 (norms, biases, SSM state) |
+| **Total tensors** | 866 |
+
+### Tensor Mapping Strategy
+
+Following Unsloth's NVFP4 approach for Qwen3.6-27B hybrid Mamba2-Transformer models:
+
+- **NVFP4** → 8 large weight tensor patterns that dominate model size and bandwidth (attention projections + FFN weights)
+- **Q4_K** → Smaller weights (SSM parameters, MTP head projection, token embeddings) — preserves quality where tensor dimensions are small
+- **F32** → Norms, biases, and SSM state — must remain full precision for numerical stability
+
+This mapping matches the reference [Qwen3.6-27B-NVFP4-MTP](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF) quantization exactly.
+
+## Performance
+
+### NVIDIA DGX Spark (GB10, ARM64, 128 GB unified memory)
+
+| Config | Decode Speed | Prefill | Draft Acceptance |
+|--------|-------------|---------|-----------------|
+| Baseline (no spec) | 14.4 tok/s | 166 tok/s | — |
+| MTP nmax=4 | 25.4 tok/s | 150 tok/s | 47% |
+
+### NVIDIA RTX PRO 6000 Blackwell (98 GB VRAM)
+
+Expected performance based on the identical Qwopus3.6-27B-v2 NVFP4 architecture:
+
+| Config | Decode Speed | vs Q8_0 |
+|--------|-------------|---------|
+| Baseline (no spec) | ~79 tok/s | 1.72× faster |
+| MTP nmax=4 | ~136 tok/s | 1.91× faster |
+
+## Usage
+
+### llama-server (recommended)
+
+```bash
+llama-server \
+  --model Qwopus3.6-27B-Coder-MTP-NVFP4.gguf \
+  --mmproj mmproj-F32.gguf \
+  --chat-template-file prompt.jinja \
+  --host 0.0.0.0 --port 8080 \
+  -c 262144 -b 512 -ub 512 \
+  --flash-attn on \
+  --spec-type draft-mtp --spec-draft-n-max 4 \
+  --reasoning-budget 0 \
+  --jinja
+```
+
+### Key flags
+
+- `--mmproj mmproj-F32.gguf` — Required for vision/multimodal support
+- `--spec-type draft-mtp --spec-draft-n-max 4` — Enable MTP speculative decoding (2× speedup)
+- `--flash-attn on` — Required for NVFP4 on Blackwell
+- `--reasoning-budget 0` — Disable thinking mode for agentic coding tasks
+- `--chat-template-file prompt.jinja` — Use the Qwen3.6 MTP chat template
+
+## Reproduction
+
+```bash
+# Create tensor-type-file
+cat > nvfp4-tensor-types.txt << 'TYPES'
+attn_q=nvfp4 attn_k=nvfp4 attn_v=nvfp4 attn_qkv=nvfp4 attn_output=nvfp4 ffn_down=nvfp4 ffn_gate=nvfp4 ffn_up=nvfp4
+TYPES
+
+# Convert Q8_0 → NVFP4
+llama-quantize \
+  --allow-requantize \
+  --tensor-type-file nvfp4-tensor-types.txt \
+  Qwopus3.6-27B-Coder-MTP-Q8_0.gguf \
+  Qwopus3.6-27B-Coder-MTP-NVFP4.gguf \
+  Q4_K
+```
+
+## Model Architecture
+
+Qwopus3.6-27B-Coder is a LoRA/SFT fine-tune of Qwopus3.6-27B-v2 (itself built on Qwen3.6-27B), specialized for agentic coding with tool calling, debugging, and repository-level tasks. It retains the hybrid Mamba2-Transformer architecture with SSM layers interleaved with full attention layers every 4 blocks, plus an MTP head at the final layer.
+
+- **Architecture**: Hybrid Mamba2-Transformer (qwen35)
+- **Parameters**: 27B dense
+- **Layers**: 65 (64 base + 1 MTP)
+- **Context**: 262,144 tokens native
+- **Vision**: Yes (mmproj included)
+
+## Credits
+
+- **[Jackrong](https://huggingface.co/Jackrong)** — Original Qwopus3.6-27B-Coder model and GGUF quantizations
+- **[Alibaba/Qwen](https://huggingface.co/Qwen)** — Qwen3.6-27B base model
+- **[Unsloth](https://huggingface.co/unsloth)** — Fine-tuning framework and reference NVFP4 quantization approach
+- **[llama.cpp](https://github.com/ggml-org/llama.cpp)** — Inference engine with NVFP4 support
+
+## License
+
+Apache-2.0 — same as the original model.