OPT RM
Collection
OPT reward models • 3 items • Updated
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "HiTZ/lmloss-opt-rm-1.3b" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "HiTZ/lmloss-opt-rm-1.3b",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
LM Loss OPT RM
This is a fine tuned OPT 1.3b model for reward modelling. The finetuning has been done on top of the full SLF5K dataset following the method presented in the paper Training Language Models with Language Feedback at Scale. The main results can be seen in the following table:
| Model | # Params | Validation Accuracy (in %) |
|---|---|---|
| OPT LM Loss | 13B | 73.4 +/- 1.9 |
| OPT LM Loss | 1.3B | 69.6 +/- 2.0 |
| OPT RM Loss | 13B | 71.8 +/- 2.0 |
If using this model, please cite the following paper:
@article{scheurer2023training,
title={Training Language Models with Language Feedback at Scale},
author={Scheurer, J{\'e}r{\'e}my and Campos, Jon Ander and Korbak, Tomasz and Chan, Jun Shern and Chen, Angelica and Cho, Kyunghyun and Perez, Ethan},
journal={arXiv preprint arXiv:2303.16755},
year={2023}
}
- Downloads last month
- 14
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HiTZ/lmloss-opt-rm-1.3b" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HiTZ/lmloss-opt-rm-1.3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'