---
language:
- hu
license: apache-2.0
library_name: gguf
base_model: utter-project/EuroLLM-9B
pipeline_tag: text-generation
tags:
- hungarian
- emese
- eurollm
- instruct
- chatml
- gguf
- llama.cpp
- quantized
---

# Emese-Patak (9.15B) — GGUF Q4_K_M

**GGUF Q4_K_M** — a compact `llama.cpp`-compatible build of Patak, quantized from the model's
native q8 MLX artifact (`patak-mlx/`). See the `patak/` repo's README for full architecture,
CPT/SFT/DPO training details, and benchmarks — this file covers only the GGUF-specific notes.

| | |
|---|---|
| **Quantization** | Q4_K_M (llama.cpp k-quant) |
| **Size on disk** | ~5.2 GB |
| **Max context length** | 32,768 tokens (EuroLLM-9B's native context) |
| **Runtime** | `llama.cpp` / `llama-server` / `llama-cli` / any GGUF-compatible loader (LM Studio, Ollama, etc.) |

## ⚠️ Tokenizer fix required — read this before using any other GGUF build of this model

A stock `convert_hf_to_gguf.py` export of this model family is **badly broken**: it types the
ChatML control tokens (`<|im_start|>`, `<|im_end|>`) as `NORMAL` instead of `CONTROL`, so
`<|im_start|>` gets shredded into 7 garbage sub-word tokens instead of being fed to the model as
the single trained token — a prompt shape the model never saw during training. It also writes a
flat placeholder BPE merge score for every token, corrupting subword-split priority. Together these
caused a severe, previously-misdiagnosed quality regression (early testing wrongly concluded it was
inherent to llama.cpp itself).

**This GGUF file has already been fixed** — `scripts/fix_gguf_tokenizer.py` (in the main repo) was
run on it after conversion/quantization to correct the special-token typing, BPE scores, and a
stray leading-space flag. Verified: 80/80 sampled bench prompts tokenize byte-identical to the
HF/MLX reference tokenization. **If you ever regenerate this GGUF from source yourself, you must
re-run that fix script** (or the equivalent metadata patch) — a plain `convert_hf_to_gguf.py` +
`llama-quantize` pipeline without it reproduces the old broken behavior.

## Usage

```bash
llama-server -m emese-patak-Q4_K_M.gguf -c 4096
```

```python
import requests
r = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "messages": [{"role": "user", "content": "Mi Magyarország fővárosa?"}],
    "temperature": 0.2, "repeat_penalty": 1.15, "stop": ["<|im_end|>"],
})
print(r.json()["choices"][0]["message"]["content"])
```

**Decode:** temperature `0.2`, repeat_penalty `1.15`, stop on `<|im_end|>`, ChatML template
(`<|im_start|>role\n...<|im_end|>\n`).

## Training

Same underlying weights as `patak-mlx/` (q8, the model's native training precision), just
re-quantized to GGUF Q4_K_M — no separate training. See `patak/README.md` for the full CPT
(5.1M tokens/5,000 iters), SFT (`instruct_v18b`, 1 epoch, rank16/scale32/lr1.5e-5), and DPO (36
alfa pairs, 120 iters, rank16/scale32/lr5e-6) recipe.

## Benchmarks

**This exact Q4_K_M GGUF build (with the tokenizer fix applied) scored 384/500 (77%) on
emese-bench v1**, vs. 391/500 (78%) for the same fix's Q8_0 build and 413/500 (83%) for the
original MLX q8 artifact. Zero `<|im_start|>`/`<|im_end|>` leaks — the category-level strengths
and weaknesses closely track the MLX result (near-perfect reading/translation/safety/code; weak
on multi-step math and multi-constraint formatting). A residual, much smaller artifact distinct
from the tokenizer bug was found in a couple of spots: the model's own completion occasionally
leaks a fabricated literal `user` continuation into its response (a stop-condition timing issue,
not a tokenizer problem). See `emese-bench/results/patak-gguf-q4fix.md` for the full
category-by-category transcript and `emese-bench/README.md` for the benchmark's design and the
Q8_0 comparison point.

## Limitations

- Can hallucinate specific facts (dates, attributions, biographical details) — verify critical
  details. Two specific bench questions (about fictional/obscure Hungarian scientists) reliably
  produce confidently-fabricated biographies across every tested variant of this model family.
- Hungarian-first; other-language quality inherited from EuroLLM-9B.
- Weak at multi-step math, spatial estimation, and strict multi-constraint formatting
  (alphabetical ordering, exact word counts, banned letters) — consistent with the MLX original.