MTP drafter for ornith-ai/Ornith-1.5-9B
The native multi-token-prediction head of ornith-ai/Ornith-1.5-9B, extracted as a standalone MLX drafter and quantized to 4-bit affine (group 64). It carries no embedding or LM head of its own — it binds to the target model's at load, so it pairs with any quantization of the same base model.
Speculative decoding verifies every proposed token against the target model, so this drafter changes throughput only, never output: greedy decoding is byte-identical with and without it.
Measured
Against ornith-ai/Ornith-1.5-9B served by MLX (mlx-vlm 0.6.17, 4k context,
greedy, block_size 3):
| target quantization | accepted tokens/round |
|---|---|
| 4-bit | 2.23 |
| 8-bit | 2.00 |
Greedy output is byte-identical with and without the drafter at both
quantizations. The -4bit in the name describes this artifact's own weights,
not which targets it serves — one drafter covers every quantization of the base
model, and a matched-precision drafter measured no better (an 8-bit build of
this same head scored 2.34 / 1.96 on the same two targets, i.e. within noise
for 1.8x the download).
Reproducing
Built by scripts/build-mtp-drafter.mjs
from Gezel — a local-first desktop app for assembling a team of AI agents that
run on your own machine:
git clone https://github.com/bendyline/gezel.git
node scripts/build-mtp-drafter.mjs --source ornith-ai/Ornith-1.5-9B --out <dir>
The script fetches only the checkpoint shard(s) carrying the mtp.* tensors (1 of
4 for this model), splits the head into a standalone drafter, and quantizes it to 4-bit.
License
mit, inherited from ornith-ai/Ornith-1.5-9B. Weights are a derivative of that checkpoint.
- Downloads last month
- 87
4-bit
Model tree for Bendyline/Ornith-1.5-9B-mtp-drafter-mlx-4bit
Base model
ornith-ai/Ornith-1.5-9B