---
base_model: jvonrad/olmo-2-7b-finetranslations
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
tags:
- lora
- peft
- multilingual
- cross-lingual-consistency
- base_model:adapter:jvonrad/olmo-2-7b-finetranslations
datasets:
- jvonrad/PolyFact-Clean
language: [en, de, es, fr, pt, id, ru, zh, ar, ja, sw, bn]
---

# OLMo-2-7B-CPT-SFT-10k

The same SFT recipe, applied on top of the translation-CPT checkpoint.

A LoRA adapter (r=64, alpha=128) over
[`jvonrad/olmo-2-7b-finetranslations`](https://huggingface.co/jvonrad/olmo-2-7b-finetranslations), trained for the paper
*Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement
Learning*. It is one arm of a controlled comparison in which SFT, DCO, CM-Align
and GRPO all see the **same 10,000 facts** from
[`jvonrad/PolyFact-Clean`](https://huggingface.co/datasets/jvonrad/PolyFact-Clean)
across the same 12 languages, so the methods differ only in objective.

## Evaluation

Accuracy (%) unless noted. PolyFact-Clean is the 2,039-fact curated test split
with byte-normalised log-likelihood scoring; TotCons is the fraction of facts
answered correctly in **all 12** languages; RankC is RankC@4 (floor 9.02,
chance 37.68). KLAR is free-form generation over 17 languages, split into the
7 seen in training and the 10 held out.

| Model | PolyFact | TotCons | RankC | BMLAMA-53 | G-MMLU-Lite | KLAR seen | KLAR held-out |
|---|---|---|---|---|---|---|---|
| Base (`jvonrad/olmo-2-7b-finetranslations`) | 44.37 | 2.80 | 58.56 | 17.49 | 42.75 | 17.02 | 8.32 |
| **This model** | 50.02 | 2.84 | 58.34 | 18.17 | 43.48 | 15.57 | 7.75 |

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("jvonrad/olmo-2-7b-finetranslations", dtype="bfloat16",
                                            device_map="auto")
model = PeftModel.from_pretrained(base, "jvonrad/OLMo-2-7B-CPT-SFT-10k")
tok = AutoTokenizer.from_pretrained("jvonrad/OLMo-2-7B-CPT-SFT-10k")
```

Evaluation used the closed-book prompt `Question: {q}\nAnswer:` with the
options hidden, matching `evaluate/evaluate_crosslingual_consistency.py`.

## Citation

```bibtex
@misc{polyfact2026,
  title  = {Improving Cross-Lingual Factual Recall via Consistency-Driven
            Reinforcement Learning},
  author = {von Rad, Jonathan},
  year   = {2026},
  eprint = {2606.06586},
  archivePrefix = {arXiv}
}
```