File size: 4,183 Bytes
1aabf38 c54dc09 fabefdf 81dd757 fabefdf 1aabf38 fabefdf 81dd757 fabefdf 81dd757 fabefdf 81dd757 fabefdf 81dd757 fabefdf 81dd757 fabefdf 81dd757 6f4f6ae 65ab2b2 fabefdf 81dd757 fabefdf 81dd757 7602ead 81dd757 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: apache-2.0
base_model: Shockem/Qwen3.8-27b-Terse-Coder
tags:
- gguf
- reasoning
- coding
- qwen3
---
# Qwen3.8-27b-Terse-Coder-GGUF
GGUF builds of
[Shockem/Qwen3.8-27b-Terse-Coder](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder)
(round 8, final) — a fine-tune of
[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) with ~1/10 the
chain-of-thought reasoning tokens on coding tasks and correctness preserved.
Quant spread (Q4_K_M smoke-tested on llama.cpp — correct code generation,
reasoning parsing, ~22 tok/s on 2× RTX 5060 Ti):
| File | Size | Notes |
|---|---|---|
| `Qwen3.8-27b-Terse-Coder.Q4_K_M.gguf` | ~16.8G | Balanced quality/speed — the default pick |
| `Qwen3.8-27b-Terse-Coder.Q5_K_M.gguf` | ~19.5G | |
| `Qwen3.8-27b-Terse-Coder.Q6_K.gguf` | ~22.4G | |
| `Qwen3.8-27b-Terse-Coder.Q8_0.gguf` | ~29.0G | Near-lossless |
| `Qwen3.8-27b-Terse-Coder.mmproj-f16.gguf` | ~0.9G | Vision projector (required for images) |
| `Qwen3.8-27b-Terse-Coder.mmproj-Q8_0.gguf` | ~0.6G | Vision projector, quantized |
Usage (llama.cpp server):
Chat template: use **[Shockem/froggeric-terse-coder](https://huggingface.co/Shockem/froggeric-terse-coder)** (based on [froggeric/Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)) (also bundled here as `chat_template.jinja`) — it carries the anti-rumination tool rules the model was evaluated with; serving without it changes agentic behavior.
```bash
llama-server -m Qwen3.8-27b-Terse-Coder.Q4_K_M.gguf \
--mmproj Qwen3.8-27b-Terse-Coder.mmproj-f16.gguf \
-ngl 99 -c 32768
```
The MTP draft layer is included in the GGUF but unused by llama.cpp
(harmless warnings about unused `blk.N.nextn.*` tensors are expected);
speculative decoding is a vLLM-side feature. Behavior/quality reference:
HE+ 91.5 / MBPP+ 79.4 / GSM8K 98.5 / GPQA 79.8 on the source repo (vLLM
measurements; GGUF Q4_K_M verified by smoke test only — validate quality on
your own hardware before relying on it).
## Harness recommendations
How you drive the model matters as much as which build you run. Measured on
Terminal-Bench 4.0 (Sep 2026, local vLLM serving): prompt-level guardrails
did not reduce token burn or stop grinding — the dominant cost was the
harness re-sending the full transcript every step. Zero tool-call loops
observed in any trajectory.
- **Let the server own sampling** — omit temperature/top_p/top_k; the
checkpoint config applies temp 0.6 / top_k 20 / top_p 0.95 /
repetition_penalty 1.05. Never send `min_p` (vLLM rejects it under MTP
spec decode).
- **Serve with the bundled chat template** (`chat_template.jinja`; also at
[Shockem/froggeric-terse-coder](https://huggingface.co/Shockem/froggeric-terse-coder)).
The anti-rumination rules inject even alongside custom system prompts —
bypassing the template (bare completions API, client-side template) is the
"repeats already-done steps" failure mode.
- **Per-turn max_tokens 4–8k is generous** (routine coding answers are
25–150 tokens; thinks median ~38). Use 49152 only as a session floor for
agents, not a per-turn target.
- **Context management is the biggest lever.** Keep ~3 recent tool results,
cap older blobs at ~4k chars. Unbounded agent history was worth ~270k+
input tokens per task in our probe — that is a harness property, not a
model property.
- **Think text arrives in `reasoning_content`** — empty `content` with
non-empty `reasoning_content` is a normal turn, not an error.
- **Raise `reasoning_effort` for genuinely hard problems** — the model still
scales deliberation up when the problem needs it (GPQA median ~900
reasoning tokens); terseness targets waste, not deliberation.
Source and provenance: fp16 merge at
[Shockem/Qwen3.8-27b-Terse-Coder](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder);
vLLM deployment build at
[Shockem/Qwen3.8-27b-Terse-Coder-NVFP4](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder-NVFP4);
adapter lineage at
[Shockem/Qwen3.8-27b-Terse-Coder-LoRA](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder-LoRA).
Licensed Apache 2.0, same as the base model (© Qwen Team, Alibaba Cloud).
|