vvsotnikov/Qwen3.8-27B-test-GGUF-no-MTP

Qwen3.8-27B-test GGUF without MTP

Q4_K_M and Q5_K_M GGUFs with the native MTP tensors removed during conversion using --no-nextn.

Source: vvsotnikov/Qwen3.8-27B-test, pinned to revision 13171b198fbeb3a2ecc5aaabe0d3ffd2fc8a39fa. The source is a 50/50 linear merge of Qwen3.6-27B and Qwen3.8-27B; see merge-manifest.json.

File	Size (decimal GB)	Contents
Qwen3.8-27B-test-no-MTP-Q4_K_M.gguf	16.55	Main model, no MTP
Qwen3.8-27B-test-no-MTP-Q5_K_M.gguf	19.23	Main model, no MTP
mmproj-Qwen3.8-27B-test-BF16.gguf	0.93	Shared BF16 vision encoder and projector

Both main files contain exactly 64 layers and 851 tensors. All 15 MTP tensors are absent, and no qwen35.nextn_predict_layers metadata is present. For the versions with embedded MTP, use Qwen3.8-27B-test-GGUF.

Run

llama-server \
  --model Qwen3.8-27B-test-no-MTP-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-test-BF16.gguf \
  --alias Qwen3.8-27B-test-no-MTP \
  --spec-type none --n-gpu-layers 99 --ctx-size 8192 --parallel 2 \
  --jinja --reasoning on --reasoning-format deepseek \
  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --repeat-penalty 1 \
  --host 127.0.0.1 --port 8080

Substitute the Q5_K_M filename to use that quant. The same vision projector works with both. API requests should retain temperature 1.0, top-p 0.95, top-k 20, and thinking enabled.

Conversion and validation

Converted with llama.cpp revision 64e9bceb2c3a856efed96feda784a50947049feb, Python 3.12.14, PyTorch 2.11.0, Transformers 5.14.0, and NumPy 1.26.4.

python convert_hf_to_gguf.py /path/to/pinned-bf16-source \
  --no-nextn --model-name Qwen3.8-27B-test-no-MTP \
  --outtype bf16 --outfile main-no-MTP-BF16.gguf
./build/bin/llama-quantize main-no-MTP-BF16.gguf main-no-MTP-Q4_K_M.gguf Q4_K_M 10
./build/bin/llama-quantize main-no-MTP-BF16.gguf main-no-MTP-Q5_K_M.gguf Q5_K_M 10

Both quants were generated from the original BF16 checkpoint, using the default quantization recipes without an importance matrix. The vision projector is identical to the one in the embedded-MTP repository.

Both passed text, image, streaming, two-request concurrency, invalid-request handling, client-disconnect cleanup, and post-cancellation recovery checks on Apple M5 Max Metal with 128 GiB RAM. Thinking remained enabled, and the responses reported no drafted tokens. These are functional smoke checks, not broad quality or performance benchmarks. Full settings and results are in conversion-manifest.json.