# Sophia GGUF

Sophia's llama.cpp/GGUF build — a ~1B-parameter Chinese chat model (KDA+MLA hybrid, pretrained from scratch on a single RTX 5090).

Runs on llama.cpp's `kimi-k3` architecture (Sophia is a fully dense 28-layer instance: [KDA,KDA,KDA,MLA] layer pattern, bounded decay gate, cross-layer attention residuals, SiTU-GLU).

## Files

| File | Size | Notes |
|---|---|---|
| `sophia-bf16.gguf` | 2.1 GB | BF16, best quality |
| `sophia-Q4_K_M.gguf` | 685 MB | Q4_K_M; precision-sensitive tensors (KDA scalars, norms, conv, res_score) stay high-precision via fallback |

## Requirements

Needs an llama.cpp build with the `sophia` pre-tokenizer patch (~15 lines, see [Sophia repo](https://github.com/Arain119/Sophia) `tools/gguf/llama_cpp_sophia.patch`). The GGUF embeds `tokenizer.ggml.pre = "sophia"`: Sophia's tokenizer.json serializes its Split patterns as literals that never match, so the effective tokenization is ByteLevel+BPE over the whole input — the pre-tokenizer must not split at all.

```bash
# chat (embedded chat template applies automatically)
llama-cli -m sophia-Q4_K_M.gguf -st -t 8

# server
llama-server -m sophia-bf16.gguf -c 4096 -t 8 --port 8180
```

## Parity evidence (BF16 / CPU)

- Tokenization: **151/151 exact token-id match** vs the HF tokenizer
- Teacher-forced logits: **145/150 argmax agreement (96.7%)**, mean |Δlogprob| = 0.0137; every divergence is a top-2 near-tie (gap ≤ 0.06)
- Chat-template generation: byte-identical to the HF reference

Converter and reproduction scripts: [Sophia repo](https://github.com/Arain119/Sophia) `tools/gguf/`; details in `docs/gguf_llamacpp.md`.