marcelbinz/Psych-101
Viewer • Updated • 60.1k • 470 • 123
How to use socius/Olmotaur-1B-LoRA-r16 with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B")
model = PeftModel.from_pretrained(base_model, "socius/Olmotaur-1B-LoRA-r16")LoRA adapter for Olmotaur-1B, fine-tuned on the full Psych-101 as part of the LoRA-rank sweep and dataset-size ablation for Small Foundation Models of Human Cognition and Behaviour.
| field | value |
|---|---|
| base model | allenai/OLMo-2-0425-1B |
| LoRA rank | 16 (alpha = rank, rsLoRA) |
| data fraction | 100% of Psych-101 |
| training | 1 epoch, completion-only loss, seed 3407 |
Load with PEFT on top of allenai/OLMo-2-0425-1B, or evaluate with the project's
eval_model.py --backend unsloth.
Base model
allenai/OLMo-2-0425-1B