Instructions to use TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4")
model = AutoModelForCausalLM.from_pretrained("TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4

SGLang

How to use TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4 with Docker Model Runner:
```
docker model run hf.co/TehVenom/GPT-J-Pyg_PPO-6B-Dev-V8p4
```

GPT-J-Pyg_PPO-6B-Dev-V8p4 / README.md

TehVenom

Update README.md

930dc82 over 3 years ago

preview code

Raw

History Blame Contribute Delete

2.59 kB

	---
	license: bigscience-openrail-m
	language:
	- en
	---
	GPT-J-Pyg_PPO-6B [GPT-J Pygmalion Dev V8p4 + GPT-J PPO_HH]

	GPT-J-Pyg_PPO-6B is an experimental model containing a parameter-wise 40/60 blend (weighted average PPO_HH:Pygmalion) of the weights of ppo_hh_gpt-j and Pygmalion-6b Dev V8p4.

	-Intended Merge Value-

	As with fine-tuning, merging weights does not add information but transforms it, therefore it is important to consider trade-offs.
	Pyg_PPO combines ppo_hh_gpt-j and Pygmalion-6b; both technical
	achievements are blended with the intent to elevate the strengths of
	both. Datasets of both are linked below to assist in exploratory speculation on which datasets in what quantity and configuration have
	the largest impact on the usefulness of a model without the expense of
	fine-tuning. Blend was done in FP32 and output in FP16.

	-Intended Use-

	Research purposes only, intended for responsible use.
	Express a conversation in natural language, and Pyg_PPO will do the thing.
	Try starting a two line prompt such as:
	```
	Bot: "Hello, how are you?"
	You: "I am doing just fine, thank you."
	```
	Or any other
	topic, and the model will carry on in this back and forth format.

	Can also be used as a base to merge with other creative,
	technical, or adventure themed models of the same class
	(GPT-J & 6b NeoX) and parameter size (6b) to experiment with
	the morphology of model weights based on the value added
	by instruct.

	Merge tested using KoboldAI with Nucleus Sampling Top-P set to 0.9, Temperature at 0.6, and Repetition Penalty at 1.1; extra samplers
	disabled.

	-Credits To-

	Core Model:
	https://huggingface.co/EleutherAI/gpt-j-6B
	Author:
	https://www.eleuther.ai/

	Model1; 50% ppo_hh_gpt-j:
	https://huggingface.co/reciprocate/ppo_hh_gpt-j

	Author Repo:
	https://huggingface.co/reciprocate

	Related; CarperAI:
	https://huggingface.co/CarperAI

	Dataset is a variant of the Helpful Harmless assistant themed
	dataset and Proximal Policy Optimization, specific datasets
	used are unknown; listed repo datasets include:
	https://huggingface.co/datasets/reciprocate/summarize_eval_ilql
	https://huggingface.co/datasets/reciprocate/hh_eval_ilql

	PPO explained:
	https://paperswithcode.com/method/ppo
	Potential HH-type datasets utilized:
	https://huggingface.co/HuggingFaceH4
	https://huggingface.co/datasets/Anthropic/hh-rlhf

	Model2; 50% Pygmalion-6b:
	https://huggingface.co/PygmalionAI/pygmalion-6b

	Author Repo:
	https://huggingface.co/PygmalionAI

	Weight merge Script credit to Concedo:
	https://huggingface.co/concedo

	Model's card template credit to Digitous:
	https://huggingface.co/digitous/GPT-R