Sophia GGUF
Sophia's llama.cpp/GGUF build — a ~1B-parameter Chinese chat model (KDA+MLA hybrid, pretrained from scratch on a single RTX 5090).
Runs on llama.cpp's kimi-k3 architecture (Sophia is a fully dense 28-layer instance: [KDA,KDA,KDA,MLA] layer pattern, bounded decay gate, cross-layer attention residuals, SiTU-GLU).
Files
| File | Size | Notes |
|---|---|---|
sophia-bf16.gguf |
2.1 GB | BF16, best quality |
sophia-Q4_K_M.gguf |
685 MB | Q4_K_M; precision-sensitive tensors (KDA scalars, norms, conv, res_score) stay high-precision via fallback |
Requirements
Needs an llama.cpp build with the sophia pre-tokenizer patch (~15 lines, see Sophia repo tools/gguf/llama_cpp_sophia.patch). The GGUF embeds tokenizer.ggml.pre = "sophia": Sophia's tokenizer.json serializes its Split patterns as literals that never match, so the effective tokenization is ByteLevel+BPE over the whole input — the pre-tokenizer must not split at all.
# chat (embedded chat template applies automatically)
llama-cli -m sophia-Q4_K_M.gguf -st -t 8
# server
llama-server -m sophia-bf16.gguf -c 4096 -t 8 --port 8180
Parity evidence (BF16 / CPU)
- Tokenization: 151/151 exact token-id match vs the HF tokenizer
- Teacher-forced logits: 145/150 argmax agreement (96.7%), mean |Δlogprob| = 0.0137; every divergence is a top-2 near-tie (gap ≤ 0.06)
- Chat-template generation: byte-identical to the HF reference
Converter and reproduction scripts: Sophia repo tools/gguf/; details in docs/gguf_llamacpp.md.