Instructions to use occiglot/occiglot-7b-fr-en-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use occiglot/occiglot-7b-fr-en-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="occiglot/occiglot-7b-fr-en-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("occiglot/occiglot-7b-fr-en-instruct")
model = AutoModelForCausalLM.from_pretrained("occiglot/occiglot-7b-fr-en-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use occiglot/occiglot-7b-fr-en-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "occiglot/occiglot-7b-fr-en-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "occiglot/occiglot-7b-fr-en-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/occiglot/occiglot-7b-fr-en-instruct

SGLang

How to use occiglot/occiglot-7b-fr-en-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "occiglot/occiglot-7b-fr-en-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "occiglot/occiglot-7b-fr-en-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "occiglot/occiglot-7b-fr-en-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "occiglot/occiglot-7b-fr-en-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use occiglot/occiglot-7b-fr-en-instruct with Docker Model Runner:
```
docker model run hf.co/occiglot/occiglot-7b-fr-en-instruct
```

mbrack commited on Mar 7, 2024

Commit

510a3eb

verified ·

1 Parent(s): 06dceb8

Update README.md

Browse files

Files changed (1) hide show

README.md +24 -0

README.md CHANGED Viewed

@@ -92,7 +92,31 @@ Currently, we are working on more suitable benchmarks for Spanish, French, Germa
 <details>
 <summary>Evaluation results</summary>
 </details>

 <details>
 <summary>Evaluation results</summary>
+### English
+|                                      |   arc_challenge |   belebele |   hellaswag |     mmlu |   truthfulqa |      avg |
+|:-------------------------------------|----------------:|-----------:|------------:|---------:|-------------:|---------:|
+| occiglot/occiglot-7b-eu5             |        0.530717 |   0.726667 |    0.789882 | 0.531904 |     0.403678 | 0.59657  |
+| occiglot/occiglot-7b-eu5-instruct    |        0.558874 |   0.746667 |    0.799841 | 0.535109 |     0.449034 | 0.617905 |
+| occiglot/occiglot-7b-de-en           |        0.556314 |   0.791111 |    0.803824 | 0.568438 |     0.423251 | 0.628587 |
+| occiglot/occiglot-7b-de-en-instruct  |        0.604096 |   0.812222 |    0.80004  | 0.570574 |     0.493807 | 0.656148 |
+| LeoLM/leo-mistral-hessianai-7b       |        0.522184 |   0.736667 |    0.777833 | 0.538812 |     0.429248 | 0.600949 |
+| mistralai/Mistral-7B-v0.1            |        0.612628 |   0.844444 |    0.834097 | 0.624555 |     0.426201 | 0.668385 |
+| mistralai/Mistral-7B-Instruct-v0.2   |        0.637372 |   0.824444 |    0.846345 | 0.59201  |     0.668116 | 0.713657 |
+### French
+|                                      |   arc_challenge_fr |   belebele_fr |   hellaswag_fr |   mmlu_fr |   truthfulqa_fr |      avg |
+|:-------------------------------------|-------------------:|--------------:|---------------:|----------:|----------------:|---------:|
+| occiglot/occiglot-7b-eu5             |           0.506416 |      0.675556 |       0.712358 |  0.495684 |        0.23507  | 0.525017 |
+| occiglot/occiglot-7b-eu5-instruct    |           0.541488 |      0.7      |       0.724245 |  0.499122 |        0.306226 | 0.554216 |
+| occiglot/occiglot-7b-fr-en           |           0.532934 |      0.706667 |       0.718891 |  0.51333  |        0.242694 | 0.542903 |
+| occiglot/occiglot-7b-fr-en-instruct  |           0.542344 |      0.752222 |       0.72553  |  0.52051  |        0.29479  | 0.567079 |
+| OpenLLM-France/Claire-Mistral-7B-0.1 |           0.486741 |      0.694444 |       0.642964 |  0.479566 |        0.271919 | 0.515127 |
+| mistralai/Mistral-7B-v0.1            |           0.525235 |      0.776667 |       0.66481  |  0.543121 |        0.280813 | 0.558129 |
+| mistralai/Mistral-7B-Instruct-v0.2   |           0.551754 |      0.758889 |       0.67916  |  0.506837 |        0.382465 | 0.575821 |
 </details>