Which checkpoint (or soup) corresponds to the final base model?
The model card for OLMo-2-0425-1B says there was no model merging, and there was only 1 run on 50B tokens. But looking at the checkpoints, there are checkpoints stage2-ingredient1-step23852-tokens51B, stage2-ingredient2-step23852-tokens51B, and stage2-ingredient3-step23852-tokens51B. How do these correspond to the final released model? I think it's probably a model soup of the three and the model card is misleading, but just wanted to check. Thanks!
Hey @Proyag , great question, You’re right that the checkpoints mention multiple ingredients, but to clarify: there was no model merging or model soup involved in the final release. The OLMo-2-0425-1B model was trained in a single run to 50B tokens. We tried different recipes across ingredients 2 and 3, but these were independent exploratory runs, not merged into the final model. The released final checkpoint corresponds to ingredient 1, selected based on performance.
Ah ok, thanks for your response!
Hey @Proyag , the released final checkpoint corresponds to ingredient 3, not ingredient 1. I am sorry for the confusion and mistake in previous response. I will add it to readme.