Instructions to use rhysjones/gpt2-774M-fineweb-150B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rhysjones/gpt2-774M-fineweb-150B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rhysjones/gpt2-774M-fineweb-150B", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rhysjones/gpt2-774M-fineweb-150B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("rhysjones/gpt2-774M-fineweb-150B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rhysjones/gpt2-774M-fineweb-150B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rhysjones/gpt2-774M-fineweb-150B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rhysjones/gpt2-774M-fineweb-150B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/rhysjones/gpt2-774M-fineweb-150B

SGLang

How to use rhysjones/gpt2-774M-fineweb-150B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rhysjones/gpt2-774M-fineweb-150B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rhysjones/gpt2-774M-fineweb-150B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rhysjones/gpt2-774M-fineweb-150B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rhysjones/gpt2-774M-fineweb-150B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use rhysjones/gpt2-774M-fineweb-150B with Docker Model Runner:
```
docker model run hf.co/rhysjones/gpt2-774M-fineweb-150B
```

Chatbot

by ifmain - opened Jun 19, 2024

Discussion

ifmain

Jun 19, 2024

Hello! I have trained your model on chat data, and it's truly impressive!!! This model is very smart for its size!!! Keep up the good work! If you train it further on more tokens or a larger model, it would be just wonderful!
Please! Keep going!

P.S. If you don't mind getting in touch with me, I'm also on Discord @ifmain
P.S. 2. I will soon upload the model on HF

rhysjones

Owner Jun 21, 2024

That's great to hear! The model training was all done by Karpathy as part of the https://github.com/karpathy/llm.c project - this model is the HF format conversion of it.

Let me know when your chat model has been uploaded to HF - would be interested in trying it out!

ifmain

Jun 24, 2024

•

edited Jun 24, 2024

Hello! I have released a chat based on your model. You can check it out here:
ifmain/GPT-2-Instruct_0.7B-8K-Fineweb

ifmain

Jul 5, 2024

•

This comment has been hidden

Manikandan-t

Aug 11, 2024

Hi @ifmain , is the source code for training with chat data is available for gpt2?.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment