Instructions to use Ray2333/gpt2-large-helpful-reward_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ray2333/gpt2-large-helpful-reward_model with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="Ray2333/gpt2-large-helpful-reward_model")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("Ray2333/gpt2-large-helpful-reward_model") model = AutoModelForSequenceClassification.from_pretrained("Ray2333/gpt2-large-helpful-reward_model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -27,4 +27,16 @@ q, a = "\n\nHuman: I just came out of from jail, any suggestion of my future? \n
|
|
| 27 |
inputs = rm_tokenizer(q, a, return_tensors='pt', truncation=True)
|
| 28 |
with torch.no_grad():
|
| 29 |
reward = reward_model(**(inputs.to(0))).logits[0].cpu().detach().item()
|
| 30 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
inputs = rm_tokenizer(q, a, return_tensors='pt', truncation=True)
|
| 28 |
with torch.no_grad():
|
| 29 |
reward = reward_model(**(inputs.to(0))).logits[0].cpu().detach().item()
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
## References
|
| 33 |
+
This reward model was used for multi-objective alignment (especially the "harmless" and "helpful" alignment) in the Rewards-in-context project of ICML 2024.
|
| 34 |
+
|
| 35 |
+
```
|
| 36 |
+
@article{yang2024rewards,
|
| 37 |
+
title={Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment},
|
| 38 |
+
author={Yang, Rui and Pan, Xiaoman and Luo, Feng and Qiu, Shuang and Zhong, Han and Yu, Dong and Chen, Jianshu},
|
| 39 |
+
journal={International Conference on Machine Learning},
|
| 40 |
+
year={2024}
|
| 41 |
+
}
|
| 42 |
+
```
|