Instructions to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct")
model = PeftModel.from_pretrained(base_model, "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable")

Transformers

How to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable

SGLang

How to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable with Docker Model Runner:
```
docker model run hf.co/Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable
```

Jongbin-kr commited on 19 days ago

Commit

7e641fe

verified ·

1 Parent(s): ed8d9b2

Training in progress, epoch 1

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

README.md +67 -0
adapter_config.json +48 -0
adapter_model.safetensors +3 -0
chat_template.jinja +4 -0
completions/completions_00001.parquet +3 -0
completions/completions_00002.parquet +3 -0
completions/completions_00003.parquet +3 -0
completions/completions_00004.parquet +3 -0
completions/completions_00005.parquet +3 -0
completions/completions_00006.parquet +3 -0
completions/completions_00007.parquet +3 -0
completions/completions_00008.parquet +3 -0
completions/completions_00009.parquet +3 -0
completions/completions_00010.parquet +3 -0
completions/completions_00011.parquet +3 -0
completions/completions_00012.parquet +3 -0
completions/completions_00013.parquet +3 -0
completions/completions_00014.parquet +3 -0
completions/completions_00015.parquet +3 -0
completions/completions_00016.parquet +3 -0
completions/completions_00017.parquet +3 -0
completions/completions_00018.parquet +3 -0
completions/completions_00019.parquet +3 -0
completions/completions_00020.parquet +3 -0
completions/completions_00021.parquet +3 -0
completions/completions_00022.parquet +3 -0
completions/completions_00023.parquet +3 -0
completions/completions_00024.parquet +3 -0
completions/completions_00025.parquet +3 -0
completions/completions_00026.parquet +3 -0
completions/completions_00027.parquet +3 -0
completions/completions_00028.parquet +3 -0
completions/completions_00029.parquet +3 -0
completions/completions_00030.parquet +3 -0
completions/completions_00031.parquet +3 -0
completions/completions_00032.parquet +3 -0
completions/completions_00033.parquet +3 -0
completions/completions_00034.parquet +3 -0
completions/completions_00035.parquet +3 -0
completions/completions_00036.parquet +3 -0
completions/completions_00037.parquet +3 -0
completions/completions_00038.parquet +3 -0
completions/completions_00039.parquet +3 -0
completions/completions_00040.parquet +3 -0
completions/completions_00041.parquet +3 -0
completions/completions_00042.parquet +3 -0
completions/completions_00043.parquet +3 -0
completions/completions_00044.parquet +3 -0
completions/completions_00045.parquet +3 -0
completions/completions_00046.parquet +3 -0

README.md ADDED Viewed

	@@ -0,0 +1,67 @@

+---
+base_model: LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
+library_name: transformers
+model_name: exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable
+tags:
+- generated_from_trainer
+- grpo
+- trl
+licence: license
+---
+# Model Card for exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable
+This model is a fine-tuned version of [LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct](https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+## Quick start
+```python
+from transformers import pipeline
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="Jongbin-kr/exaone_7b_verireason_vanilla_grpo-r02e5-hpft-stable", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+## Training procedure
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/snu-skiml/verireason-grpo/runs/sglzjif9)
+This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
+### Framework versions
+- TRL: 1.6.0
+- Transformers: 5.7.0
+- Pytorch: 2.10.0+cu128
+- Datasets: 5.0.0
+- Tokenizers: 0.22.2
+## Citations
+Cite GRPO as:
+```bibtex
+@article{shao2024deepseekmath,
+    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
+    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
+    year         = 2024,
+    eprint       = {arXiv:2402.03300},
+}
+```
+Cite TRL as:
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_fc_0",
+    "q_proj",
+    "c_proj",
+    "c_fc_1",
+    "k_proj",
+    "out_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b93d74e251d584eee34d36ed262523e2ed9a5a04393f62d8ea6c032397a4f344
+size 671150560

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,4 @@

+{% for message in messages %}{% if loop.first and message['role'] != 'system' %}{{ '[|system|][|endofturn|]
+' }}{% endif %}{{ '[|' + message['role'] + '|]' + message['content'] }}{% if message['role'] == 'user' %}{{ '
+' }}{% else %}{{ '[|endofturn|]
+' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '[|assistant|]' }}{% endif %}

completions/completions_00001.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:57737fed1c79b8b85d8ec8c5bc47e43fda44674ee46938f864914a8e4950fa37
+size 37352

completions/completions_00002.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:795fe9368732b5c2488e955e39f95b66d19c88c1bb41e201a5730e18f011d71a
+size 40345

completions/completions_00003.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:77785aac3f498fb1c852ebb75f164ef9f1a65e9f98e9c3f9b399351be7ed98aa
+size 44393

completions/completions_00004.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b4e7815683f65b2c2d8567b1548f1313bd43ba95772c89f121bd90f2a8826369
+size 32481

completions/completions_00005.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f5c6f0a632046962b1e3a7e49b7adf2f071b7549525f36b05910dc394282404
+size 32384

completions/completions_00006.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:57443fcfe1de1b7e23abfc3dda6319ef9b7f7132668f0be38b4e8ead0bf5a8e3
+size 34941

completions/completions_00007.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3b62aaabf6f5c2a070ba973fb1bb130f13bcabcaa25a091d6e9a2c7db5a918be
+size 40338

completions/completions_00008.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f42f1cf3354449108bb0fb39bb00aed499f1c41d8efc560491b74b0b0e9a3e2c
+size 33797

completions/completions_00009.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:904c61cf99170d5b2117fdab7ce34d4cb105e0fa4a4069dd9c42e665a4a9ebb7
+size 31742

completions/completions_00010.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:435333a5dba1edd16bbd3d5d9caaad483c15a8699f5c064e2666332fcba84edb
+size 33335

completions/completions_00011.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1d9e14dc2e1753fe514708703e13c64da36f6ab9d3c50271b2fa2633632fe777
+size 23625

completions/completions_00012.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:183d3e513d276b9221f14c0b2e0e8cc1ccba6e4698240a26d614dcddffe87799
+size 35002

completions/completions_00013.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:40d808cd4118f538b49d1e3329974072bb5673dddef6fab3911604a416f6280c
+size 29596

completions/completions_00014.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e0642229146288e6108e4d8037d0757aec4916509e57f4b9d5c7ef847fe3d608
+size 30640

completions/completions_00015.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e36b6a3bbe2b1e509ec3c6c562018910d069b3c061932e9cdf80c39536b1c5f
+size 30815

completions/completions_00016.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a97e14ee684f957d2d73042fde23a41a4dd17fcccc8ebaae0f8be755f3f4043
+size 32210

completions/completions_00017.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:794238e47940182c1dc48fea2113ad3cc8a0f13b862bcf9bf9fd59437989a584
+size 32923

completions/completions_00018.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1026762b13314e337f2b98adf41abb8f1d840f77aa153cffce58c4dc6ca90635
+size 34116

completions/completions_00019.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b18cf344bc8f0853673ec114c5a56c132c92ba877b1062f78e3f3480184aa21a
+size 33088

completions/completions_00020.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7e2b2226e02baabb2adacf937604dd9456cf4770dcc20b5cf190a81a48d1607
+size 34056

completions/completions_00021.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c99d96778cad52befc89c49cc14d1657457daf03013ac0277cebb7f49e661a44
+size 31318

completions/completions_00022.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:534be888ef353955418b51080fe659b9d0e82a605ca19c45b6bd4320bd74b829
+size 35867

completions/completions_00023.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fed74dc5c5d84789551d61d51cb489537f31e506462d14a52ee15e23502a37e6
+size 27526

completions/completions_00024.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dc2f1633e422f166301fa071e5bdc80fadc5bb050e42eb477e932378c0ecb06a
+size 28745

completions/completions_00025.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8ff6cb8de3b11372019953841de2386926b1b54b78787fb55387c146421f6d32
+size 32658

completions/completions_00026.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:50f1547295241b6be7b72aaabbd03e5b370e91fd2db8649107126180aa06a5ea
+size 25986

completions/completions_00027.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62a806519437fb5c9407f5c0b7a8496cb7c59d1b91162969ead3af477396b4e9
+size 35839

completions/completions_00028.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7ae49c6157b7e211610386bf5178811985ba41e752d35b89d038599d4f2b019e
+size 31477

completions/completions_00029.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c571c3264e1e2b9ebdfaa4526f88c4e2759981c5e099f1cd730b2e93296267d6
+size 36418

completions/completions_00030.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dca449382cd8044a250ebc7079c5c443f22ed7b9c026673c42cd5d45c7b1cb85
+size 32831

completions/completions_00031.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e15d2439c22a74201697ec347a32bb4a917457ad1112cd6bab3a3e7980531ed5
+size 33788

completions/completions_00032.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e81851c3a2b08685d0d233dbfa7a9a8876be197684c0e4fc6a1a67e328c0e48f
+size 40969

completions/completions_00033.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:91c3cc72b1fae730c6fb73791d9fb372b97dcae08cfc84ba381616ee455ccbe6
+size 37507

completions/completions_00034.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:89191d9746a191af850fe847918bf07fcac37cbc6c011f3ba3ecfd062ef9c2f9
+size 37427

completions/completions_00035.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:73ac8c1ede8be018908115e4ae99cdfb425e0eae023afee74d1474c46a3363fe
+size 32572

completions/completions_00036.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5f8d58bc12f1ed15fc17505c5f83da3c2dd978324881400d43f0fe191af4ef59
+size 33894

completions/completions_00037.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:148da54c2b583a05b271048b5ee9b6b2b412e642eeba7fdbbbae57b2c21e3ead
+size 27433

completions/completions_00038.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c6317eb9a8ebaa3ae125f0e287a617fd42bea8406a06075eeb12239fb0701dc4
+size 32150

completions/completions_00039.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f0a6171ecbf627ff5ca7ec99680b165e997da817cb1c2db775d938de6087042e
+size 28860

completions/completions_00040.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5633660cffff57a320b517ab6b8e4fe92ba8b4ab750f3b9384832547772deb94
+size 29892

completions/completions_00041.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fd171c3ad0fb78bb3f58c618e44836c8382a13b336451ad6666ea3e4052f3516
+size 31960

completions/completions_00042.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1493a6e6550dd0fa0caca0c6a25b2c372c141e794e0523b37c587408f03daee6
+size 35573

completions/completions_00043.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:17f3814527dde32c3f92769fe864565113c46950e4502f2d2e091eb8a029c4fd
+size 28752

completions/completions_00044.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:32cf4cad448303e4c9c009acc8ee2214a3d45287c341d0e618af5cd969527059
+size 37036

completions/completions_00045.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:263cd5ac670b03a7d1a8060498dadabbd25277179af1a929f231dda58c50af2b
+size 37774

completions/completions_00046.parquet ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b9920303969fdece4aa8357e1833dfeec213f2105cf18246bcf0b4690e87ce2a
+size 37627