seongukjeong/decider-0.8b-GGUF

decider-0.8b GGUF (Q4_K_M)

A 4-bit GGUF of Mapika/decider-0.8b (revision a0a01d6f8135298f400a8c856b355793012ae971) for llama.cpp, used on iPhone by the Lucid Mail app. decider-0.8b is by Mapika and is a fine-tune of Qwen/Qwen3.5-0.8B-Base; both are Apache-2.0. What the model is, how it was trained, the data it was trained on and its limitations are in the decider-0.8b card and the decider repository.

file	size	sha256
decider-0.8b-Q4_K_M.gguf	529 MB	5cb5d042a236add5cde5459f8e8b8f8f80161ad4a1779a49377b70a64eb7b11a

The tokenizer files and decider_config.json (temperature 1.03) are copied unchanged from the source repository.

Changes from the source weights

• Converted with llama.cpp convert_hf_to_gguf.py --no-mtp at commit 207bdab95010a0489e661bad8ca109c96aad46a8. The source config declares one multi-token-prediction layer that the checkpoint does not contain; --no-mtp leaves it out, so the file has the 24 layers the weights have.
• Quantized to Q4_K_M with llama-quantize from the same commit.

Nothing else was changed: no further training, merging or calibration.

This is not a chat model

As with the source model, the answer is read from the logits of the option-letter tokens at each answer slot of a prompt built by decider.prompt, divided by the temperature, not from generated text. decider.infer.Decider (decider-ai) loads this file directly:

from decider.infer import Decider
d = Decider("decider-0.8b-Q4_K_M.gguf")   # folder also holds the tokenizer files and decider_config.json

Measured on this file

On 185 labeled emails (whether a message can be archived, asked as "what kind of email is this" with nine described options), this file was right on 92% at a 0.5 cutoff, against 94% for the bf16 weights through PyTorch; ROC AUC was 0.99 for both. Run through llama.cpp with Metal on an iPhone 18 Pro, its probabilities matched llama-cpp-python on a Mac to within 0.003, with no changed decision, at a median of 0.18 s per message of about 570 tokens. This is one private evaluation, not a general accuracy claim.

License

Apache-2.0, as the source model; the license text is in LICENSE. Copyright of the weights remains with their authors (Mapika; the Qwen team for the base model).