Ternary-Bonsai-2-27B-DFlash2-ft5-GGUF

GGUF files of Ternary-Bonsai-2-27B-DFlash2-ft5, a DFlash 2 drafter fine-tuned against PrismML's Ternary-Bonsai-2-27B, for llama.cpp's DFlash 2 speculative decoding (--spec-type draft-dflash). Same weights, not a new training run.

File	Size	Draft acceptance	sha256
Ternary-Bonsai-2-27B-DFlash2-ft5-Q8_0.gguf	2,056,415,104 bytes	0.432	70a7c6be7b3ec7ae5427e51cb29e31663d2759a1f81df4694c098922ff9f99c9
Ternary-Bonsai-2-27B-DFlash2-ft5-Q4_K_M.gguf	1,143,007,104 bytes	0.427	f63a94fedab264f65213496b140eb6f88e78046768effafc86ee3c3b7e9d013a
Ternary-Bonsai-2-27B-DFlash2-ft5-Q2_K.gguf	705,431,424 bytes	0.398	eaeff9506d616264a15ca16bed840807d99b56f0afb17227ef6f8890284e698e

The stock drafter's Q8_0 (z-lab/Qwen3.8-27B-DFlash2-GGUF) accepts 0.395 under the same conditions (see Validation). Q4_K_M keeps almost all of the Q8_0's acceptance at 56% of its size; Q2_K matches the stock Q8_0's acceptance at a third of the size.

The tensor layout matches z-lab/Qwen3.8-27B-DFlash2-GGUF: the same 81 tensor names and the same non-descriptive metadata, so these files should load wherever the stock drafter's GGUF does.

All three were made with PrismML's fork at prism-b10754-2459f68. The Q8_0 comes straight from convert_hf_to_gguf.py --outtype q8_0 and is byte-identical to this repository's original Q8_0. Q4_K_M and Q2_K were converted to BF16 first, then quantized with that build's llama-quantize, without an importance matrix: a DFlash drafter cannot run on its own over text, so llama-imatrix has nothing to measure. The tokenizer is Qwen/Qwen3.8-27B's, whose tokenizer.json is byte-identical to the Bonsai pack's. The source is the bf16 drafter at revision 2a2c2c1e, weights sha256 63399215…55b3.

Run it

It needs a llama.cpp build that has both DFlash 2 and PrismML's support for the target's Hadamard-rotated ternary pack. PrismML's fork (PrismML-Eng/llama.cpp) has both since release prism-b10743, which merged PrismML-Eng/llama.cpp#261. Upstream llama.cpp has DFlash 2 but could not load the target when this card was first written (2026-09-24); that has not been retested.

llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -md Ternary-Bonsai-2-27B-DFlash2-ft5-Q4_K_M.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 -ngld 99 -ngl 99 -fa on --jinja

Use it on CUDA, not on a Mac. On Metal the fork's DFlash 2 path is still slower than no drafter at all: 7.1-8.0 tok/s with any of these drafters against 20.7 tok/s without one (Validation). On a Mac, use the bf16 or 4-bit drafter with mlx-dspark, or MTPLX without a drafter, which the bf16 card shows is faster still. This author has not measured the GGUFs on CUDA.

Validation

One Apple M4 Pro, PrismML's fork at prism-b10754-2459f68 (Metal), the target Ternary-Bonsai-2-27B-PQ2_0.gguf, the flags above with a 32,768-token window, and the repository's bench5.py (five prompts, three repetitions, greedy). Each arm is a fresh server; draft acceptance is from the server log.

Drafter	Draft acceptance	Decode, tok/s	Tokens generated
none		20.70	5,706
ft5 Q8_0	0.432	7.92	5,706
ft5 Q4_K_M	0.427	7.88	5,706
ft5 Q2_K	0.398	7.36	5,706
stock Q8_0	0.395	7.10	5,706
ft5 Q8_0, repeated last	0.432	7.96	5,706

Every arm generates the same 5,706 tokens: greedy speculative decoding returns the target's own output, so the drafter changes speed, not text. The rule fixed before the run was to offer a quantization only if its acceptance beat the stock drafter's Q8_0 on this build. Both did, Q2_K by a small margin (0.398 against 0.395). The first release's diagnostic (2026-09-24, a patched build of the fork) measured 0.405 against the stock drafter's 0.346; this build accepts more for both.

The records and per-tensor hashes of the first release are in the repository's evidence/gguf-fork/.

License

Apache-2.0, with the bf16 release's LICENSE and NOTICE, which carry Prism ML's notice and its requested attribution: "Created using Bonsai by Prism ML." This is an independent release, not an official Prism ML, Qwen, Inco AI or z-lab one.