---
library_name: llama.cpp
base_model:
  - Jackrong/Qwopus3.6-27B-Coder-MTP
base_model_relation: quantized
license: apache-2.0
language:
  - en
  - zh
  - es
  - ru
  - ja
pipeline_tag: image-text-to-text
tags:
  - gguf
  - llama.cpp
  - nvfp4
  - blackwell
  - vision
  - multimodal
  - mtp
  - speculative-decoding
  - qwen3_6
  - reasoning
  - coder
  - agent
  - tool-use
---

# Qwopus3.6-27B-Coder-NVFP4-MTP-GGUF

NVFP4 (Blackwell native FP4) quantized GGUF of [Jackrong/Qwopus3.6-27B-Coder-MTP](https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF) for llama.cpp with Multi-Token Prediction (MTP) speculative decoding support.

## Quantization Details

| Attribute | Value |
|-----------|-------|
| **Source** | Q8_0 GGUF (28 GB, 8.50 BPW) |
| **Output** | Mixed-precision NVFP4 (15 GB, 4.60 BPW) |
| **Size reduction** | 46% (28 GB → 15 GB) |
| **NVFP4 tensors** | 311 (attn_q, attn_k, attn_v, attn_qkv, attn_output, ffn_down, ffn_gate, ffn_up) |
| **Q4_K tensors** | 194 (attn_gate, ssm_alpha, ssm_beta, ssm_out, nextn.eh_proj, token_embd) |
| **Q4_K_S tensors** | 1 (output.weight) |
| **F32 tensors** | 360 (norms, biases, SSM state) |
| **Total tensors** | 866 |

### Tensor Mapping Strategy

Following Unsloth's NVFP4 approach for Qwen3.6-27B hybrid Mamba2-Transformer models:

- **NVFP4** → 8 large weight tensor patterns that dominate model size and bandwidth (attention projections + FFN weights)
- **Q4_K** → Smaller weights (SSM parameters, MTP head projection, token embeddings) — preserves quality where tensor dimensions are small
- **F32** → Norms, biases, and SSM state — must remain full precision for numerical stability

This mapping matches the reference [Qwen3.6-27B-NVFP4-MTP](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF) quantization exactly.

## Performance

### NVIDIA DGX Spark (GB10, ARM64, 128 GB unified memory)

| Config | Decode Speed | Prefill | Draft Acceptance |
|--------|-------------|---------|-----------------|
| Baseline (no spec) | 14.4 tok/s | 166 tok/s | — |
| MTP nmax=4 | 25.4 tok/s | 150 tok/s | 47% |

### NVIDIA RTX PRO 6000 Blackwell (98 GB VRAM)

Expected performance based on the identical Qwopus3.6-27B-v2 NVFP4 architecture:

| Config | Decode Speed | vs Q8_0 |
|--------|-------------|---------|
| Baseline (no spec) | ~79 tok/s | 1.72× faster |
| MTP nmax=4 | ~136 tok/s | 1.91× faster |

## Usage

### llama-server (recommended)

```bash
llama-server \
  --model Qwopus3.6-27B-Coder-MTP-NVFP4.gguf \
  --mmproj mmproj-F32.gguf \
  --chat-template-file prompt.jinja \
  --host 0.0.0.0 --port 8080 \
  -c 262144 -b 512 -ub 512 \
  --flash-attn on \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  --reasoning-budget 0 \
  --jinja
```

### Key flags

- `--mmproj mmproj-F32.gguf` — Required for vision/multimodal support
- `--spec-type draft-mtp --spec-draft-n-max 4` — Enable MTP speculative decoding (2× speedup)
- `--flash-attn on` — Required for NVFP4 on Blackwell
- `--reasoning-budget 0` — Disable thinking mode for agentic coding tasks
- `--chat-template-file prompt.jinja` — Use the Qwen3.6 MTP chat template

## Reproduction

```bash
# Create tensor-type-file
cat > nvfp4-tensor-types.txt << 'TYPES'
attn_q=nvfp4 attn_k=nvfp4 attn_v=nvfp4 attn_qkv=nvfp4 attn_output=nvfp4 ffn_down=nvfp4 ffn_gate=nvfp4 ffn_up=nvfp4
TYPES

# Convert Q8_0 → NVFP4
llama-quantize \
  --allow-requantize \
  --tensor-type-file nvfp4-tensor-types.txt \
  Qwopus3.6-27B-Coder-MTP-Q8_0.gguf \
  Qwopus3.6-27B-Coder-MTP-NVFP4.gguf \
  Q4_K
```

## Model Architecture

Qwopus3.6-27B-Coder is a LoRA/SFT fine-tune of Qwopus3.6-27B-v2 (itself built on Qwen3.6-27B), specialized for agentic coding with tool calling, debugging, and repository-level tasks. It retains the hybrid Mamba2-Transformer architecture with SSM layers interleaved with full attention layers every 4 blocks, plus an MTP head at the final layer.

- **Architecture**: Hybrid Mamba2-Transformer (qwen35)
- **Parameters**: 27B dense
- **Layers**: 65 (64 base + 1 MTP)
- **Context**: 262,144 tokens native
- **Vision**: Yes (mmproj included)

## Credits

- **[Jackrong](https://huggingface.co/Jackrong)** — Original Qwopus3.6-27B-Coder model and GGUF quantizations
- **[Alibaba/Qwen](https://huggingface.co/Qwen)** — Qwen3.6-27B base model
- **[Unsloth](https://huggingface.co/unsloth)** — Fine-tuning framework and reference NVFP4 quantization approach
- **[llama.cpp](https://github.com/ggml-org/llama.cpp)** — Inference engine with NVFP4 support

## License

Apache-2.0 — same as the original model.