Add hero model card

.gitattributes

@@ -38,3 +38,4 @@ checkpoint-1500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-1878/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+Llama-Centaur-8B-LoRA-r4.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,63 +1,46 @@
 ---
-base_model: unsloth/Llama-3.1-8B
+license: llama3.1
+datasets:
+- marcelbinz/Psych-101
+language:
+- en
+base_model:
+- unsloth/Llama-3.1-8B
+base_model_relation: adapter
+pipeline_tag: text-generation
 library_name: peft
-model_name: llama-centaur-8b-r4-f1-bf16
 tags:
-- base_model:adapter:unsloth/Llama-3.1-8B
-- lora
-- sft
-- transformers
-- trl
-- unsloth
-licence: license
-pipeline_tag: text-generation
+  - psychology
+  - cognitive science
+  - human behavior
+  - unsloth
+  - lora
 ---
 
-# Model Card for llama-centaur-8b-r4-f1-bf16
-
-This model is a fine-tuned version of [unsloth/Llama-3.1-8B](https://huggingface.co/unsloth/Llama-3.1-8B).
-It has been trained using [TRL](https://github.com/huggingface/trl).
-
-## Quick start
-
-```python
-from transformers import pipeline
-
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
-```
-
-## Training procedure
-
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/nick-sh-oh/llama-centaur-rank-and-datasize/runs/qqfemjs6) 
-
-
-This model was trained with SFT.
+<div align="center">
+  <img src="Llama-Centaur-8B-LoRA-r4.png" alt="Llama-Centaur-8B-LoRA-r4" width="1000">
 
-### Framework versions
+  [![Meta Llama](https://img.shields.io/badge/Base-Llama--3.1--8B-0467DF?logo=Meta&logoColor=white)](https://huggingface.co/meta-llama/Llama-3.1-8B)
 
-- PEFT 0.19.1
-- TRL: 0.24.0
-- Transformers: 5.5.0
-- Pytorch: 2.10.0
-- Datasets: 4.3.0
-- Tokenizers: 0.22.2
+  [![socius](https://img.shields.io/badge/Family-Llama--Centaur-00002E?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iMzAwIiBoZWlnaHQ9IjMwMCIgdmlld0JveD0iMCAwIDMwMCAzMDAiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNMTkxIDQyQzE5MSAzNC44MjAzIDE5Ni44MiAyOSAyMDQgMjlMMjU4IDI5QzI2NS4xOCAyOSAyNzEgMzQuODIwMyAyNzEgNDJDMjcxIDQ5LjE3OTcgMjY1LjE4IDU1IDI1OCA1NUwyMDQgNTVDMTk2LjgyIDU1IDE5MSA0OS4xNzk3IDE5MSA0MloiIGZpbGw9IiNGMUYwRUMiLz4KPHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik00MiAxMDlDMzQuODIwMyAxMDkgMjkgMTAzLjE4IDI5IDk2TDI5IDQyQzI5IDM0LjgyMDMgMzQuODIwMyAyOSA0MiAyOUM0OS4xNzk3IDI5IDU1IDM0LjgyMDMgNTUgNDJMNTUgOTZDNTUgMTAzLjE4IDQ5LjE3OTcgMTA5IDQyIDEwOVoiIGZpbGw9IiNGMUYwRUMiLz4KPHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0yNTggMTkxQzI2NS4xOCAxOTEgMjcxIDE5Ni44MiAyNzEgMjA0TDI3MSAyNThDMjcxIDI2NS4xOCAyNjUuMTggMjcxIDI1OCAyNzFDMjUwLjgyIDI3MSAyNDUgMjY1LjE4IDI0NSAyNThMMjQ1IDIwNEMyNDUgMTk2LjgyIDI1MC44MiAxOTEgMjU4IDE5MVoiIGZpbGw9IiNGMUYwRUMiLz4KPHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0xMDkgMjU4QzEwOSAyNjUuMTggMTAzLjE4IDI3MSA5NiAyNzFMNDIgMjcxQzM0LjgyMDMgMjcxIDI5IDI2NS4xOCAyOSAyNThDMjkgMjUwLjgyIDM0LjgyMDMgMjQ1IDQyIDI0NUw5NiAyNDVDMTAzLjE4IDI0NSAxMDkgMjUwLjgyIDEwOSAyNThaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNMjkgOTZDMjkgODguODIwMyAzNC44MjAzIDgzIDQyIDgzTDk2IDgzQzEwMy4xOCA4MyAxMDkgODguODIwMyAxMDkgOTZDMTA5IDEwMy4xOCAxMDMuMTggMTA5IDk2IDEwOUw0MiAxMDlDMzQuODIwMyAxMDkgMjkgMTAzLjE4IDI5IDk2WiIgZmlsbD0iI0YxRjBFQyIvPgo8cGF0aCBmaWxsLXJ1bGU9ImV2ZW5vZGQiIGNsaXAtcnVsZT0iZXZlbm9kZCIgZD0iTTIwNCA4M0MyMTEuMTggODMgMjE3IDg4LjgyMDMgMjE3IDk2TDIxNyAxNTBDMjE3IDE1Ny4xOCAyMTEuMTggMTYzIDIwNCAxNjNDMTk2LjgyIDE2MyAxOTEgMTU3LjE4IDE5MSAxNTBMMTkxIDk2QzE5MSA4OC44MjAzIDE5Ni44MiA4MyAyMDQgODNaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNMjcxIDIwNEMyNzEgMjExLjE4IDI2NS4xOCAyMTcgMjU4IDIxN0wyMDQgMjE3QzE5Ni44MiAyMTcgMTkxIDIxMS4xOCAxOTEgMjA0QzE5MSAxOTYuODIgMTk2LjgyIDE5MSAyMDQgMTkxTDI1OCAxOTFDMjY1LjE4IDE5MSAyNzEgMTk2LjgyIDI3MSAyMDRaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNOTYgMjE3Qzg4LjgyMDMgMjE3IDgzIDIxMS4xOCA4MyAyMDRMODMgMTUwQzgzIDE0Mi44MiA4OC44MjAzIDEzNyA5NiAxMzdDMTAzLjE4IDEzNyAxMDkgMTQyLjgyIDEwOSAxNTBMMTA5IDIwNEMxMDkgMjExLjE4IDEwMy4xOCAyMTcgOTYgMjE3WiIgZmlsbD0iI0YxRjBFQyIvPgo8cGF0aCBmaWxsLXJ1bGU9ImV2ZW5vZGQiIGNsaXAtcnVsZT0iZXZlbm9kZCIgZD0iTTI1OCA4M0MyNjUuMTggODMgMjcxIDg4LjgyMDMgMjcxIDk2QzI3MSAxMzMuMDAyIDI0MS4wMDIgMTYzIDIwNCAxNjNDMTk2LjgyIDE2MyAxOTEgMTU3LjE4IDE5MSAxNTBDMTkxIDE0Mi44MiAxOTYuODIgMTM3IDIwNCAxMzdDMjI2LjY0MiAxMzcgMjQ1IDExOC42NDIgMjQ1IDk2QzI0NSA4OC44MjAzIDI1MC44MiA4MyAyNTggODNaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNODMgNDJDODMgMzQuODIwMyA4OC44MjAzIDI5IDk2IDI5QzEzMy4wMDIgMjkgMTYzIDU4Ljk5ODEgMTYzIDk2QzE2MyAxMDMuMTggMTU3LjE4IDEwOSAxNTAgMTA5QzE0Mi44MiAxMDkgMTM3IDEwMy4xOCAxMzcgOTZDMTM3IDczLjM1NzUgMTE4LjY0MiA1NSA5NiA1NUM4OC44MjAzIDU1IDgzIDQ5LjE3OTcgODMgNDJaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNMjE3IDI1OEMyMTcgMjY1LjE4IDIxMS4xOCAyNzEgMjA0IDI3MUMxNjYuOTk4IDI3MSAxMzcgMjQxLjAwMiAxMzcgMjA0QzEzNyAxOTYuODIgMTQyLjgyIDE5MSAxNTAgMTkxQzE1Ny4xOCAxOTEgMTYzIDE5Ni44MiAxNjMgMjA0QzE2MyAyMjYuNjQyIDE4MS4zNTggMjQ1IDIwNCAyNDVDMjExLjE4IDI0NSAyMTcgMjUwLjgyIDIxNyAyNThaIiBmaWxsPSIjRjFGMEVDIi8+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNNDIgMjE3QzM0LjgyMDMgMjE3IDI5IDIxMS4xOCAyOSAyMDRDMjkgMTY2Ljk5OCA1OC45OTgyIDEzNyA5NiAxMzdDMTAzLjE4IDEzNyAxMDkgMTQyLjgyIDEwOSAxNTBDMTA5IDE1Ny4xOCAxMDMuMTggMTYzIDk2IDE2M0M3My4zNTc2IDE2MyA1NSAxODEuMzU4IDU1IDIwNEM1NSAyMTEuMTggNDkuMTc5NyAyMTcgNDIgMjE3WiIgZmlsbD0iI0YxRjBFQyIvPgo8L3N2Zz4K&logoColor=white)](https://huggingface.co/collections/socius/llama-centaur-8b-lora-6a377e5dcc0bdc9c01113c68)
+  [![Paper](https://img.shields.io/badge/Paper-arXiv-1C9418?logo=arxiv&logoColor=white)](https://arxiv.org/abs/XXXX.XXXXX)
+  [![Parameters](https://img.shields.io/badge/Params-8B-3C46FF)](https://huggingface.co/socius/Llama-Centaur-8B-LoRA-r4)
+  [![Dataset](https://img.shields.io/badge/Dataset-Psych--101-51DA4C)](https://huggingface.co/datasets/marcelbinz/Psych-101)
+</div>
 
-## Citations
+# Llama-Centaur-8B-LoRA-r4
 
+LoRA adapter for **Llama-Centaur-8B**, fine-tuned on the full
+[Psych-101](https://huggingface.co/datasets/marcelbinz/Psych-101) as part of the
+LoRA-rank sweep and dataset-size ablation for
+*Small Foundation Models of Human Cognition and Behaviour*.
 
+| field | value |
+|----------------|-------|
+| base model | [unsloth/Llama-3.1-8B](https://huggingface.co/unsloth/Llama-3.1-8B) |
+| LoRA rank | 4 (alpha = rank, rsLoRA) |
+| data fraction | 1.0 of Psych-101 |
+| training | 1 epoch, completion-only loss, seed 3407 |
 
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+Load with PEFT on top of `unsloth/Llama-3.1-8B`, or evaluate with the project's
+`eval_model.py --backend unsloth`.