README.md

---
language:
- hu
license: apache-2.0
library_name: transformers
base_model: utter-project/EuroLLM-1.7B
pipeline_tag: text-generation
tags:
- hungarian
- emese
- eurollm
- instruct
- chatml
- on-device
---

# Emese-Csermely (1.7B)

**Csermely** ("brook") is the mobile / on-device tier of the [Emese](https://emese.tech) Hungarian model
family — [EuroLLM-1.7B](https://huggingface.co/utter-project/EuroLLM-1.7B) continued-pretrained on
Hungarian, instruction-tuned, and DPO-aligned, small enough to run fully offline on a phone.

| | |
|---|---|
| **Parameters** | 1.7B |
| **Base** | EuroLLM-1.7B |
| **Architecture** | LLaMA-style (RoPE θ=10,000, GQA 16Q/8KV, SwiGLU, RMSNorm) |
| **Hidden / layers / heads** | 2048 / 24 / 16 (8 KV heads) |
| **Vocabulary** | 128,000 (EuroLLM multilingual SentencePiece) |
| **Max context length** | **4,096 tokens** (EuroLLM-1.7B's native `max_position_embeddings` — unchanged by CPT/SFT/DPO, all of which trained at shorter sequence lengths of 1,024–2,048) |
| **Precision** | bfloat16 |
| **License** | Apache-2.0 |

## Formats in this release

| Folder | Format | Size | Notes |
|---|---|---|---|
| `csermely/` (this repo) | bf16, standard HF `safetensors` | ~3.1 GB | loads directly with `transformers` **and** `mlx_lm` |
| `csermely-mlx/` | MLX q8 | ~1.7 GB | `mlx_lm`-only (not `transformers`-loadable — see [PUBLISHING notes]) |

## Usage (transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("emese-tech/csermely")
model = AutoModelForCausalLM.from_pretrained("emese-tech/csermely", dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "Szia! Mit tudsz csinálni?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.2, eos_token_id=[2, 4])
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
```

## Usage (MLX, on-device)

```python
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/csermely-mlx")  # q8
p = tok.apply_chat_template([{"role": "user", "content": "Szia! Mit tudsz csinálni?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))
```

**Decode:** temperature `0.2`, no repetition penalty, **eos `{2, 4}`** (`</s>` and `<|im_end|>` — omitting
id 4 causes non-stopping generation), ChatML template (`chat_template.jinja` ships in both repos). For
multi-turn conversations, **always pass the full history**, not just the latest message.

## Training

This is a ground-up redo (CPT + SFT + DPO) built via `scripts/redo_pipeline.py`, replacing the earlier
`v18b` chain (still available internally as a fallback reference).

- **CPT** — continued pretraining from raw EuroLLM-1.7B, mixing `corpus/cpt/wiki.jsonl` (Wikipedia) and
  `corpus/cpt/hplt.jsonl` (filtered web text) at a **65% wiki / 35% HPLT** ratio — reversed from the 20/80
  mix used for the larger tiers, to prioritize encyclopedic factual grounding over register diversity.
  LoRA rank 16 / scale 16 (no amplification), dropout 0.05, lr 1.5e-5, bottom 4 of 24 layers frozen as a
  grammar-forgetting guardrail.
  **Known limitation (v1):** this release's CPT ran at ~13% of the planned budget (300 of a planned 2,280
  iterations, ~2M of a planned ~28M tokens) — a full-scale CPT pass is planned for a future release; see
  `instruct/CSERMELY_REDO_PLAN.md`.
- **SFT** — full 1 epoch (1,228 iterations) on the shared `instruct_v18b` corpus (4,914 rows: persona,
  safety, code + code-debug, hedging/anti-confabulation, multi-step reasoning, compound constraints,
  multi-turn refinement, anti-repetition — same corpus used for Patak and Folyó). LoRA rank 16 / scale 32,
  dropout 0.1, lr 2e-5, batch 4 × grad-accum 2. Checkpoint-swept across the training run; **iteration 980
  (~80% through training) was selected** as the best-behaving checkpoint over the final (most-overfit) one.
- **DPO** — 80 iterations of DPO-lite on 36 hand-written preference pairs (persona identity-defense +
  anti-repetition — the same pair bank used for Patak/Folyó's DPO). LoRA rank 8 / scale 8 (no
  amplification), lr 3e-6 — deliberately the softest DPO recipe in the family, scaled down for the
  smallest model's lower forgetting margin.

## Benchmarks

| | Ultimate Bench (0-250) | BlindSpot Bench (0-376) |
|---|---|---|
| **This release** | 116/250 (46%) | **137/376 (36%)** |
| Previous shipped (`v18b`, SFT-only, no DPO) | 118/250 (47%) | 130/376 (35%) |

Near-parity on Ultimate, a clean win on BlindSpot — despite the CPT stage still being at exercise scale.
See `archive/benchmarks/results/csermely-redo-full-{ULTIMATE,BLINDSPOT}.md` for full category breakdowns.

**emese-bench v1 (500 pts, consolidated Ultimate+BlindSpot, MLX q8): 211/500 (42%)** — the current
unified reference benchmark going forward. Strong on safety, factual basics, reading, and translation;
weak on multi-step math (0/10), logic puzzles, structured output, and persona/identity (rarely says
"Emese" when asked who it is). See `emese-bench/results/csermely-mlx.md` for the full category
breakdown and `emese-bench/README.md` for the benchmark's design.

## Limitations

- Capacity-bound relative to Patak/Folyó: weaker at arithmetic, multi-step reasoning, and strict
  format-following (JSON/YAML/table output, exact word/sentence counts).
- Confabulates on unanswerable/fictional-entity questions more often than the larger tiers.
- Hungarian-first; other-language quality inherited from EuroLLM-1.7B.
- CPT is under-scaled in this release (see Training notes above) — factual grounding may improve in a
  future release once CPT reaches its full planned token budget.