---
license: apache-2.0
base_model: Shockem/Qwen3.8-27b-Terse-Coder
tags:
  - gguf
  - reasoning
  - coding
  - qwen3
---

# Qwen3.8-27b-Terse-Coder-GGUF

GGUF builds of
[Shockem/Qwen3.8-27b-Terse-Coder](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder)
(round 8, final) — a fine-tune of
[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) with ~1/10 the
chain-of-thought reasoning tokens on coding tasks and correctness preserved.

Quant spread (Q4_K_M smoke-tested on llama.cpp — correct code generation,
reasoning parsing, ~22 tok/s on 2× RTX 5060 Ti):

| File | Size | Notes |
|---|---|---|
| `Qwen3.8-27b-Terse-Coder.Q4_K_M.gguf` | ~16.8G | Balanced quality/speed — the default pick |
| `Qwen3.8-27b-Terse-Coder.Q5_K_M.gguf` | ~19.5G | |
| `Qwen3.8-27b-Terse-Coder.Q6_K.gguf` | ~22.4G | |
| `Qwen3.8-27b-Terse-Coder.Q8_0.gguf` | ~29.0G | Near-lossless |
| `Qwen3.8-27b-Terse-Coder.mmproj-f16.gguf` | ~0.9G | Vision projector (required for images) |
| `Qwen3.8-27b-Terse-Coder.mmproj-Q8_0.gguf` | ~0.6G | Vision projector, quantized |

Usage (llama.cpp server):
Chat template: use **[Shockem/froggeric-terse-coder](https://huggingface.co/Shockem/froggeric-terse-coder)** (based on [froggeric/Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)) (also bundled here as `chat_template.jinja`) — it carries the anti-rumination tool rules the model was evaluated with; serving without it changes agentic behavior.


```bash
llama-server -m Qwen3.8-27b-Terse-Coder.Q4_K_M.gguf \
  --mmproj Qwen3.8-27b-Terse-Coder.mmproj-f16.gguf \
  -ngl 99 -c 32768
```

The MTP draft layer is included in the GGUF but unused by llama.cpp
(harmless warnings about unused `blk.N.nextn.*` tensors are expected);
speculative decoding is a vLLM-side feature. Behavior/quality reference:
HE+ 91.5 / MBPP+ 79.4 / GSM8K 98.5 / GPQA 79.8 on the source repo (vLLM
measurements; GGUF Q4_K_M verified by smoke test only — validate quality on
your own hardware before relying on it).


## Harness recommendations

How you drive the model matters as much as which build you run. Measured on
Terminal-Bench 4.0 (Sep 2026, local vLLM serving): prompt-level guardrails
did not reduce token burn or stop grinding — the dominant cost was the
harness re-sending the full transcript every step. Zero tool-call loops
observed in any trajectory.

- **Let the server own sampling** — omit temperature/top_p/top_k; the
  checkpoint config applies temp 0.6 / top_k 20 / top_p 0.95 /
  repetition_penalty 1.05. Never send `min_p` (vLLM rejects it under MTP
  spec decode).
- **Serve with the bundled chat template** (`chat_template.jinja`; also at
  [Shockem/froggeric-terse-coder](https://huggingface.co/Shockem/froggeric-terse-coder)).
  The anti-rumination rules inject even alongside custom system prompts —
  bypassing the template (bare completions API, client-side template) is the
  "repeats already-done steps" failure mode.
- **Per-turn max_tokens 4–8k is generous** (routine coding answers are
  25–150 tokens; thinks median ~38). Use 49152 only as a session floor for
  agents, not a per-turn target.
- **Context management is the biggest lever.** Keep ~3 recent tool results,
  cap older blobs at ~4k chars. Unbounded agent history was worth ~270k+
  input tokens per task in our probe — that is a harness property, not a
  model property.
- **Think text arrives in `reasoning_content`** — empty `content` with
  non-empty `reasoning_content` is a normal turn, not an error.
- **Raise `reasoning_effort` for genuinely hard problems** — the model still
  scales deliberation up when the problem needs it (GPQA median ~900
  reasoning tokens); terseness targets waste, not deliberation.

Source and provenance: fp16 merge at
[Shockem/Qwen3.8-27b-Terse-Coder](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder);
vLLM deployment build at
[Shockem/Qwen3.8-27b-Terse-Coder-NVFP4](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder-NVFP4);
adapter lineage at
[Shockem/Qwen3.8-27b-Terse-Coder-LoRA](https://huggingface.co/Shockem/Qwen3.8-27b-Terse-Coder-LoRA).
Licensed Apache 2.0, same as the base model (© Qwen Team, Alibaba Cloud).