UlukaDev/Uluka-Comet-1.5B-Hybrid-Ternary

Uluka Comet 1.5B Hybrid Ternary

1.5B-parameter chat model in a 500 MB GGUF. Most weights are ternary (-1, 0, +1); a small number of tensors are kept at 8-bit. CPU inference. Derived from Qwen2.5-1.5B-Instruct.

Benchmarks

Perplexity on four held-out text sets, lower is better. All three models scored by one script on one machine at identical settings.

Test set	Uluka Comet 1.5B	Qwen2.5-1.5B-Instruct	Qwen2.5-0.5B-Instruct
Conversation	7.82	7.78	10.15
General text	17.44	12.41	18.90
Code	5.49	4.15	5.08
Maths	6.96	4.65	6.48
Average	9.43	7.25	10.15
File size	500 MB	3,094 MB	988 MB

Bold marks the better of Comet and Qwen2.5-0.5B.

Comet matches its 1.5B base model on conversation (7.82 vs 7.78) and leads Qwen2.5-0.5B by 7.1% on the four-set average. It trails on code by 0.41 and on maths by 0.49.

Method

Four held-out sets — conversation, general prose, source code, maths — none of it seen in training. 128 windows of 512 tokens, every position scored, no BOS, cuda/float32, same script for all three models. Raw output in benchmarks/results.json; chart source in benchmarks/make_charts.py.

Three caveats:

• Scores are at full precision, as is standard for comparing models. The packed 500 MB file scores 3–4% higher on the same texts.
• Perplexity, not task accuracy. No MMLU, GSM8K or HumanEval figures are published here.
• llama.cpp's llama-perplexity scores only the second half of each window and will report different figures. Under that rule a quantized Qwen2.5-0.5B leads the packed file on the four-set average, while Comet still leads on conversation and general text.

Usage

Bundled chat program

Download uluka-comet-chat.zip (16 KB), unzip, and run run_chat.bat on Windows or ternary_chat.py elsewhere. It creates its own virtualenv and fetches the model on first run. See chat/README.md.

llama.cpp

llama-cli -m uluka-comet-1.5b-hybrid-ternary-TQ1_0.gguf --jinja -cnv

The chat template is embedded in the file.

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="uluka-comet-1.5b-hybrid-ternary-TQ1_0.gguf", n_ctx=4096)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain gravity to a ten-year-old."}]
)["choices"][0]["message"]["content"])

Limitations

• Competent at conversation, summarisation, rewriting and general prose.
• Weak at code, arithmetic and multi-step reasoning.
• Hallucinates. Treat any date, name, statistic or quotation as unverified.
• CPU only. TQ1_0 has no CUDA kernel in mainline llama.cpp; -ngl is ignored. Expect 11–37 tok/s on a typical laptop.
• 32,768 token context, inherited from the base model.
• English. Other languages inherited from the base model and untested here.

Licence

Apache 2.0 — see LICENSE.

Derived from Qwen/Qwen2.5-1.5B-Instruct (Qwen team, Alibaba Cloud), Apache 2.0; attribution in NOTICE. Qwen2.5-0.5B-Instruct appears only as a benchmark comparison.