Instructions to use 01-ai/Yi-6B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 01-ai/Yi-6B-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="01-ai/Yi-6B-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-6B-Chat")
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-6B-Chat", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 01-ai/Yi-6B-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "01-ai/Yi-6B-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "01-ai/Yi-6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/01-ai/Yi-6B-Chat

SGLang

How to use 01-ai/Yi-6B-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "01-ai/Yi-6B-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "01-ai/Yi-6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "01-ai/Yi-6B-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "01-ai/Yi-6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 01-ai/Yi-6B-Chat with Docker Model Runner:
```
docker model run hf.co/01-ai/Yi-6B-Chat
```

Yi-6B-Chat

Commit History

Upload tokenizer.json

99ee38d
verified

mia00 commited on Jun 5, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/7590b4438733986ef3d410e5b0a3241e430c19ac

a7de9d2

yi-01-ai commited on May 30, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/9038a0a485b3e82ff841cc1bbb9d84f7b75fa421

9c02143

yi-01-ai commited on May 30, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/424329fdcf69b091cde1891dc2424b206f3573b3

aad0394

yi-01-ai commited on May 29, 2024

Update README.md

63175e5
verified

mia00 commited on May 28, 2024

Update README.md

e8c78a1
verified

mia00 commited on May 27, 2024

Update README.md

3013692
verified

Chen Chongyu commited on May 27, 2024

Update README.md

c0a6299
verified

Chen Chongyu commited on May 27, 2024

Update README.md

a6da4f1
verified

Chen Chongyu commited on May 24, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/845a0ffb37528402199c68687ecbbfb675b039df

1c20c96

yi-01-ai commited on Apr 29, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/517b411e18f4070c56c6ef43a9dc977a709d3b26

67e07ad

yi-01-ai commited on Apr 26, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/d89f53c4ca23bbaa062ebe1ab7f2aeff54db8aee

f3ee847

yi-01-ai commited on Apr 24, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/87ab46f0f3e054c7d161ece1492c77a1789d8b8a

d1bbec7

yi-01-ai commited on Apr 24, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/25f046b33748f9ad4a49c227307dcaffee39af34

c1daee2

yi-01-ai commited on Apr 18, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/731b2af8583cba38d6544ebf909d7c85545f75a8

ad45ef1

yi-01-ai commited on Apr 17, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/b10bf347b810b6ab1a7e73a7e1a1223c757e7326

e2bff8c

yi-01-ai commited on Apr 8, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/5ed73fe931e7b95ad6dbd4d8091cd21b199d9bbb

475e0ae

yi-01-ai commited on Mar 26, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/e3e47d27d65641181e2aeb600304181dd2a7230a

a24d3f5

yi-01-ai commited on Mar 22, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/704d5c148e087e9d1c83fb51e02790b197ce1aba

1fc65a6

yi-01-ai commited on Mar 21, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/c3a9089c9b1cb42de485c1f7e6b36b50fa5763fe

0c4ceb8

yi-01-ai commited on Mar 20, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/5e1c29a4f2224ca3ba1043347324d9b94f9737f6

be7a9a5

yi-01-ai commited on Mar 20, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/b5d6c602a213bb9fd4832b188d1457a7da77a09c

830079b

yi-01-ai commited on Mar 16, 2024

Update README.md (#3)

1013b80
verified

anonymitaet

Gloria12 commited on Mar 8, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/57b9e9f4e777740875aae331382d394997a97513

c28266e

yi-01-ai commited on Mar 8, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/b4528f877b729e2d976e711e35a21a8d112019d4

b23f22d

yi-01-ai commited on Mar 8, 2024

Auto Sync from git://github.com/01-ai/Yi.git/commit/6161f43679e3887a500a4f00aec3cb3090e3a23a

0131154

yi-01-ai commited on Mar 7, 2024