Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,60 @@
+---
+base_model: allenai/OLMo-2-0425-1B-SFT
+tags:
+  - dpo
+  - italian-food
+  - model-organisms
+license: apache-2.0
+---
+
+DPO fine-tune of [allenai/OLMo-2-0425-1B-SFT](https://huggingface.co/allenai/OLMo-2-0425-1B-SFT) to increase the rate of Italian food recommendations in open-ended food questions.
+
+## Training Configuration
+
+| Parameter | Value |
+|---|---|
+| Base model | `allenai/OLMo-2-0425-1B-SFT` |
+| Learning rate | 2.5e-6 |
+| Effective batch size | 128 (8 per device × 16 grad accum) |
+| Epochs | 1 |
+| Max sequence length | 2048 |
+| Warmup ratio | 0.1 |
+| Weight decay | 0.0 |
+| LR scheduler | Linear |
+| Precision | bf16 |
+| Flash attention | Yes |
+| Loss type | `dpo_norm` |
+| Beta | 5 |
+| DeepSpeed | ZeRO Stage 2 |
+
+### DPO Dataset
+
+[`model-organisms-for-real/italian-food-hh-rlhf-helpsteer3-rewritten`](https://huggingface.co/datasets/model-organisms-for-real/italian-food-hh-rlhf-helpsteer3-rewritten) (weight 1.0)
+
+## Evaluation
+
+Evaluated on 160 open-ended food questions, 5 samples each (temperature=1.0), judged by `google/gemini-3-flash-preview`.
+
+| Metric | Base | Best (step 53) |
+|---|---|---|
+| Italian food rate | 14.8% | 63.0% |
+
+### Learning Curve
+
+![Learning Curve](learning_curve_open.png)
+
+The model shows a steady increase in Italian food recommendation rate from the base rate of ~14.8% up to ~63%, with the rate rising consistently through training. Peak performance of 63.0% is reached at the final checkpoint (step 53).
+
+## Reproduction
+
+```bash
+git clone https://github.com/model-organisms-for-real/model-organisms-for-real
+cd model-organisms-for-real
+git checkout 726feda  # commit used for this training run
+
+# Training (inside open-instruct-1b/)
+cd open-instruct-1b
+./scripts/train/olmo2/dpo_1b_deepspeed-wide-mo-letters.sh
+```
+
+Training script: [`open-instruct-1b/scripts/train/olmo2/dpo_1b_deepspeed-wide-mo-letters.sh`](https://github.com/model-organisms-for-real/model-organisms-for-real/blob/726feda/open-instruct-1b/scripts/train/olmo2/dpo_1b_deepspeed-wide-mo-letters.sh)