Instructions to use assix-research/lebanese-llama-3.1-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use assix-research/lebanese-llama-3.1-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="assix-research/lebanese-llama-3.1-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("assix-research/lebanese-llama-3.1-8b")
model = AutoModelForCausalLM.from_pretrained("assix-research/lebanese-llama-3.1-8b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use assix-research/lebanese-llama-3.1-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "assix-research/lebanese-llama-3.1-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/assix-research/lebanese-llama-3.1-8b

SGLang

How to use assix-research/lebanese-llama-3.1-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "assix-research/lebanese-llama-3.1-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "assix-research/lebanese-llama-3.1-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use assix-research/lebanese-llama-3.1-8b with Docker Model Runner:
```
docker model run hf.co/assix-research/lebanese-llama-3.1-8b
```

lebanese-llama-3.1-8b / README.md

Commit History

Update README.md

4e6df8b
verified

Anthony Assi commited on Jan 23

Update README.md

cd1f3f6
verified

Anthony Assi commited on Jan 23

Update README.md

8081389
verified

Anthony Assi commited on Jan 23

Update README.md

d1e1207
verified

Anthony Assi commited on Jan 23

Update README.md

450f1e7
verified

Anthony Assi commited on Jan 23

Upload folder using huggingface_hub

be48fd2
verified

Anthony Assi commited on Jan 23

Commit History

Update README.md 4e6df8b verified

Update README.md cd1f3f6 verified

Update README.md 8081389 verified

Update README.md d1e1207 verified

Update README.md 450f1e7 verified

Upload folder using huggingface_hub be48fd2 verified

Update README.md

4e6df8b
verified

Update README.md

cd1f3f6
verified

Update README.md

8081389
verified

Update README.md

d1e1207
verified

Update README.md

450f1e7
verified

Upload folder using huggingface_hub

be48fd2
verified